Spectrally And Spatially Multiplexed Fluorescent Probes For In Situ Cell Labeling

ABSTRACT

Methods are provided to identify spatially and spectrally multiplexed probes in a biological environment. Such probes are identified by the ordering and color of fluorophores of the probes. The devices and methods provided facilitate determination of the locations and colors of such fluorophores, such that a probe can be identified. In some embodiments, probes are identified by applying light from a target environment to a spatial light modulator that can be used to control the direction and magnitude of chromatic dispersion of the detected light; multiple images of the target, corresponding to multiple different spatial light modulator settings, can be deconvolved and used to determine the colors and locations of fluorophores. In some embodiments, light from a region of the target can be simultaneously imaged spatially and spectrally. Correlations between the spatial and spectral images over time can be used to determine the color of fluorophores in the target.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/270,530, filed Dec. 21, 2015; U.S. Provisional Patent ApplicationNo. 62/342,270 filed May 27, 2016; U.S. Provisional Patent ApplicationNo. 62/320,681, filed Apr. 11, 2016; U.S. Provisional Patent ApplicationNo. 62/342,268, filed May 27, 2016; U.S. Provisional Patent ApplicationNo. 62/342,252, filed May 27, 2016; and U.S. Provisional PatentApplication No. 62/342,256, filed May 27, 2016, which are herebyincorporated by reference in their entirety.

BACKGROUND

Unless otherwise indicated herein, the materials described in thissection are not prior art to the claims in this application and are notadmitted to be prior art by inclusion in this section.

A variety of methods exist to image biological tissues or othermaterials at the micro-scale (i.e., at scales at or smaller than a fewmicrometers). Such methods can include optical microscopy according to avariety of different illumination schemes and using optical systemsconfigured in a variety of different ways. Samples to be imaged could bebroadly illuminated (e.g., in bright-field microscopy), exposed to somestructured illumination (e.g., light sheet microscopy), exposed topolarized illumination (e.g., phase contrast microscopy), exposed toillumination at one or more specified points (e.g., confocalmicroscopy), or illuminated according to some other scheme. Conversely,light can be received and/or focused from the samples to be imaged in avariety of ways; light can be received from a wide field of the sampleand focused on an imager, subjected to an aperture (e.g., an aperturecorresponding to an aperture used to illuminate the sample as in, e.g.,confocal microscopy) before being imaged by an imager or light sensor,or received by some other means. Further, light of different wavelengthscan be used to illuminate a sample (e.g., to excite a fluorophore in thesample) and/or light of different wavelengths can be detected from thesample to determine spectrographic information (e.g., emission spectra,excitation spectra, absorbance spectra) about the sample or according tosome other application.

SUMMARY

A variety of systems and methods are provided to microscopically image asample (e.g., a sample of biological tissue) in such a way that theidentity of probes present in the sample can be determined. Such probesinclude two or more fluorophores having respective spectral properties(e.g., colors, emission spectra, absorption spectra) and respectiverelative locations within the probe such that the identity of the probecan be determined based on detected spectral properties (e.g., colors)and relative locations of fluorophores in the sample. A great number ofdifferent probes could be used to tag respective different contents of asample (e.g., to tag different proteins, different DNA or RNA sequences,to tag different protein isoforms). By creating the probes to include anumber of different fluorophores and/or to include two, three, or morefluorophores arranged according to a specified shape, relative location,and/or ordering the number of uniquely identifiable probes could be verylarge, e.g., exponentially and/or combinatorically related to a numberof different fluorophores used or a maximum number of fluorophorespresent in each probe.

Some embodiments of the present disclosure provide a system including:(i) a light sensor that includes a plurality of light-sensitive elementsdisposed on a focal surface of the light sensor; (ii) a spatial lightmodulator that includes a reflective layer disposed beneath a refractivelayer and that is operable to have a refractive index that variesspatially across the spatial light modulator according to a controllablegradient, wherein at least the direction and magnitude of thecontrollable gradient are electronically controllable, and wherein therefractive layer is chromatically dispersive; (iii) an optical system;and (iv) a controller that is operably coupled to the light sensor andthe spatial light modulator and that is operable to perform controlleroperations. The optical system (1) directs light emitted from a targettoward the spatial light modulator and (2) directs light emitted fromthe target and reflected from the spatial light modulator to the lightsensor such that the focal surface of the light sensor is conjugate to afocal surface passing through the target. The controller operationsinclude: (i) controlling the spatial light modulator such that at leastone of the direction or magnitude of the controllable gradient aredifferent during each of a plurality of periods of time; (ii)generating, using the light sensor, a plurality of images of the target,wherein each image corresponds to a respective one of the plurality ofperiods of time; (iii) determining, based on the plurality of images,locations and colors of two or more fluorophores in the target; and (iv)determining, based on the determined colors and locations of the two ormore fluorophores, an identity of a probe that is located in the targetand that includes the two or more fluorophores.

Some embodiments of the present disclosure provide a system including:(i) a first light sensor that includes a plurality of light-sensitiveelements disposed on a focal surface of the first light sensor; (ii) asecond light sensor that includes a plurality of light-sensitiveelements; (iii) a chromatically dispersive element; (iv) an opticalsystem; and (v) a controller that is operably coupled to the first lightsensor and the second light sensor and that is operable to performcontroller operations. The optical system (1) directs light emitted froma particular region of a target to the first light sensor such that thefocal surface of the first light sensor is conjugate to a focal surfacepassing through the particular region of the target, (2) directs lightemitted from the particular region of the target toward thechromatically dispersive element, and (3) directs light emitted from theparticular region of the target that has interacted with thechromatically dispersive element to the second light sensor such thatlight of different wavelengths that is emitted from the particularregion of the target is received by corresponding differentlight-sensitive elements of the second light sensor. The controlleroperations include: (i) generating, using the plurality oflight-sensitive elements of the first light sensor, a first plurality ofrespective time-varying waveforms of light emitted from respectivedifferent locations of the particular region of the target; (ii)generating, using the plurality of light-sensitive elements of thesecond light sensor, a second plurality of respective time-varyingwaveforms of light emitted from the particular region of the target atrespective different wavelengths; (iii) determining correlations betweentime-varying waveforms of the first plurality of time-varying waveformsand time-varying waveforms of the second plurality of time-varyingwaveforms; (iv) determining, based on the determined correlations,locations and colors of two or more fluorophores in the target; and (v)determining, based on the determined colors and locations of the two ormore fluorophores, an identity of a probe that is located in the targetand that includes the two or more fluorophores.

Some embodiments of the present disclosure provide a method including:(i) controlling a spatial light modulator such that at least one of thedirection or magnitude of a controllable gradient of a refractive indexof a refractive layer of the spatial light modulator are differentduring each of a plurality of periods of time, wherein the spatial lightmodulator includes a reflective layer disposed beneath the refractivelayer and is operable to have a refractive index that varies spatiallyacross the spatial light modulator according to a controllable gradient,wherein at least the direction and magnitude of the controllablegradient are electronically controllable, and wherein the refractivelayer is chromatically dispersive; (ii) generating, using a light sensorthat includes a plurality of light-sensitive elements disposed on afocal surface of the light sensor, a plurality of images of a target,wherein each image corresponds to a respective one of the plurality ofperiods of time, wherein light that is emitted from the target istransmitted to the light sensor via an optical system, wherein theoptical system (1) directs light emitted from the target toward thespatial light modulator and (2) directs light emitted from the targetand reflected from the spatial light modulator to the light sensor suchthat the focal surface of the light sensor is conjugate to a focalsurface passing through the target; (iii) determining, based on theplurality of images, locations and colors of two or more fluorophores inthe target; and (iv) determining, based on the determined colors andlocations of the two or more fluorophores, an identity of a probe thatis located in the target and that includes the two or more fluorophores.

Some embodiments of the present disclosure provide a method including:(i) generating, using a plurality of light-sensitive elements of a firstlight sensor that are disposed on a focal surface of the first lightsensor, a first plurality of respective time-varying waveforms of lightthat is emitted from respective different locations of a particularregion of a target and transmitted to the light sensor via an opticalsystem, wherein the optical system provides the emitted light from thetarget to the first light sensor such that the focal surface of thefirst light sensor is conjugate to a focal surface passing through theparticular region of the target; (ii) generating, using a plurality oflight-sensitive elements of a second light sensor, a second plurality ofrespective time-varying waveforms of light at different respectivewavelengths that is emitted from the particular region of the target andtransmitted to the light sensor via the optical system, wherein theoptical system provides the emitted light from the target to achromatically dispersive element, wherein the optical system providesthe emitted light from the target that has interacted with thechromatically dispersive element to the second light sensor such thatlight of different wavelengths that is emitted from the particularregion of the target is received by corresponding differentlight-sensitive elements of the second light sensor; (iii) determiningcorrelations between time-varying waveforms of the first plurality oftime-varying waveforms and time-varying waveforms of the secondplurality of time-varying waveforms; (iv) determining, based on thedetermined correlations, locations and colors of two or morefluorophores in the target; and (v) determining, based on the determinedcolors and locations of the two or more fluorophores, an identity of aprobe that is located in the target and that includes the two or morefluorophores.

These as well as other aspects, advantages, and alternatives, willbecome apparent to those of ordinary skill in the art by reading thefollowing detailed description, with reference where appropriate to theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example probe.

FIG. 2A illustrates an example probe that could be imaged.

FIG. 2B illustrates an example image of the probe of FIG. 2A.

FIG. 2C illustrates an example image of the probe of FIG. 2A.

FIG. 2D illustrates an example image of the probe of FIG. 2A.

FIG. 3A illustrates a cross-section view of elements of an examplespatial light modulator.

FIG. 3B illustrates reflection of light by the spatial light modulatorof FIG. 3A.

FIG. 3C illustrates the dependence of refractive index on wavelength oflight of materials that could be incorporated into the spatial lightmodulator of FIG. 3A.

FIG. 4 illustrates an example imaging apparatus.

FIG. 5A illustrates an example probe that could be imaged.

FIG. 5B illustrates an example spatial image of the probe of FIG. 5A.

FIG. 5C illustrates example time-varying waveforms of light.

FIG. 5D illustrates an example spectral image of a particular region ofa target that includes the probe of FIG. 5A.

FIG. 5E illustrates example time-varying waveforms of light.

FIG. 6 illustrates an example imaging apparatus.

FIG. 7 illustrates an example imaging apparatus.

FIG. 8 is a functional block diagram of an example imaging system.

FIG. 9 is a flowchart of an example method.

FIG. 10 is a flowchart of an example method.

FIG. 11 is a graphical illustration of an example scheme for acquiring ahyperspectral-imaging dataset.

FIG. 12 is a graphical illustration of another example scheme foracquiring a hyperspectral-imaging dataset.

FIG. 13 is a schematic representation of an example hyperspectralimaging system.

FIG. 14 is a schematic representation of another example hyperspectralimaging system.

FIG. 15 is a schematic representation of another example hyperspectralimaging system.

FIG. 16 is a schematic representation of another example hyperspectralimaging system.

FIG. 17 is a schematic representation of another example hyperspectralimaging system.

FIG. 18 is a schematic representation of another example hyperspectralimaging system.

FIG. 19 is a schematic representation of another example hyperspectralimaging system.

FIG. 20 is a schematic representation of another example hyperspectralimaging system.

FIG. 21 is a schematic representation of another example hyperspectralimaging system.

FIG. 22 is a schematic representation of an example diffractive element.

FIG. 23 is a schematic representation of another example diffractiveelement.

FIG. 24 is a flowchart of an example method for hyperspectral imaging.

FIG. 25 is a schematic representation of an example confocal imagingsystem.

FIG. 26 is a graphical illustration for an example scheme for performinghyperspectral confocal imaging.

FIG. 27 is a flowchart of an example method for obtaining a confocalimage.

FIG. 28 is a schematic perspective representation of an example opticalsystem.

FIG. 29 is a schematic cross-sectional representation of an examplenon-deviating dispersive element.

FIG. 30 is a graphical cross-sectional illustration of an exampleoptical beam passing through the example optical system of FIG. 28.

FIG. 31 is a graphical cross-sectional illustration of another exampleoptical beam passing through the example optical system of FIG. 28.

FIG. 32 is a diagram of an optical simulation result of dispersiongenerated by the example optical system of FIG. 28.

FIG. 33 is a diagram of another optical simulation result of dispersiongenerated by the example optical system of FIG. 28.

FIG. 34 is a flowchart of an example method for dispersing an opticalbeam.

FIG. 35 is a schematic representation of an example microscope system.

FIG. 36 is a schematic representation of an example chromatic objectivelens.

FIG. 37 is a flowchart of an example method for simultaneously obtainingan image in multiple planes with an axially chromatic lens.

FIG. 38 is a schematic representation of an example system for filteringan optical beam.

FIG. 39 is a schematic representation of an example spectral slicingmodule for filtering an optical beam.

FIG. 40A is a graphical illustration of two example passbands of twoexample spectral slicing modules.

FIG. 40B is a graphical illustration of another two example passbands oftwo example spectral slicing modules.

FIG. 40C is a graphical illustration of yet another two examplepassbands of two example spectral slicing modules.

FIG. 41 is a graphical illustration of an example spectrum of an inputoptical beam entering the example system of FIG. 38.

FIG. 42A is a graphical illustration of an example spectrum of an outputoptical beam exiting the example system of FIG. 38.

FIG. 42B is a graphical illustration of another example spectrum of anoutput optical beam exiting the example system of FIG. 38.

FIG. 43 is a flowchart of an example method for filtering an opticalbeam.

DETAILED DESCRIPTION

In the following detailed description, reference is made to theaccompanying figures, which form a part hereof. In the figures, similarsymbols typically identify similar components, unless context dictatesotherwise. The illustrative embodiments described in the detaileddescription, figures, and claims are not meant to be limiting. Otherembodiments may be utilized, and other changes may be made, withoutdeparting from the scope of the subject matter presented herein. It willbe readily understood that the aspects of the present disclosure, asgenerally described herein, and illustrated in the figures, can bearranged, substituted, combined, separated, and designed in a widevariety of different configurations, all of which are explicitlycontemplated herein.

Further, while embodiments disclosed herein make reference to use on orin conjunction with samples of tissue extracted from a human body, it iscontemplated that the disclosed methods, systems and devices may be usedin any environment where spectrographic imaging and/or opticalsectioning of other tissues or other objects or elements of anenvironment is desired. The environment may be any living or non-livingbody or a portion thereof, a work piece, an implantable device, amineral, an integrated circuit, a microelectromechanical device, etc.

I. OVERVIEW

It can be advantageous to introduce contrast agents or other probes intoa target of interest (e.g., a biological sample) to facilitate imagingof specific elements of the target, e.g., to facilitate the localizationand/or determine the concentration of specific chemicals, proteins, RNAor DNA segments, or other analytes in a target environment. Such probescan be optically distinct, e.g., can differ from each other with respectto an optical excitation spectrum, an optical emission spectrum, anabsorption spectrum, a fluorescence lifetime or other temporal dynamicsof energy absorption and re-emission/reflection, an acousticalabsorption spectrum, or some other properties such that multipledifferent probes can be independently detected in an environment,facilitating the simultaneous detection of corresponding differentanalytes in the environment.

The number of distinct probes available to label and image anenvironment can limit the number of analytes that can be simultaneouslyimaged. Such a limitation could be related to a number of fluorescentlabels that are available and that are compatible with an environment ofinterest, a spectral or temporal resolving ability and/or noise level ofan imaging apparatus, a linewidth and/or tenability of an imaging lightsource, or some other factors related to the set of probes available toimage an environment and/or an imaging apparatus provided to image theenvironment.

In order to increase the number of different probes that are availableto facilitate imaging of respective different analytes in anenvironment, probes can be fabricated that each include multipledifferent fluorophores (e.g., multiple different quantum dots, Ramandyes, fluorescent proteins, fluorescent small molecules, or otherfluorophores) arranged in a specified pattern and/or order. Byincreasing the number of fluorophores present on each probe and/or byincreasing the number of distinguishable types of fluorophores (e.g., anumber of different colors of fluorophores used), the number ofdistinguishably different probes can be increased. Such different probescan be functionalized to bind to or otherwise interact with respectivedifferent analytes (e.g., by including respective different antibodies,aptamers, complimentary DNA or RNA sequences, proteins, receptors, orother binding agents), facilitating simultaneous imaging of suchdifferent analytes.

Identification of such spatially and spectrally multiplexed probes caninclude detecting the color and relative location of fluorophores of theprobes. Detecting the color of a fluorophore could include detecting aspectrum of light emitted from the fluorophore, a wavelength of lightemitted from the fluorophore, an intensity of light emitted from thefluorophore within one or more ranges of wavelengths, or detecting someother characteristic of the wavelength-dependence of the light emittedfrom the fluorophore. Additionally or alternatively, the intensity,phase, color, spectrum, or other properties of light emitted from thefluorophore in response to illumination by light at differentwavelengths could be detected and used to determine a color, anexcitation spectrum, a Raman spectrum, or some other identifyingspectral information about the fluorophore. This detection can includethe detection of spectral information about fluorophores that areseparated by very small distances (e.g., by less than approximately 50nanometers) according to the size and configuration of the probes.

In some examples, such detection of spectral information (e.g., todetermine the color of fluorophores of a probe) can include applyinglight received from an environment that includes the probe (e.g., lightemitted from the environment responsive to illumination of theenvironment) to optical elements that have one or morewavelength-dependent optical properties. In some examples, this couldinclude applying the emitted light to one or more dichroic mirrors,diffraction gratings, or other wavelength-selective reflective elements,and imaging multiple different portions of the emitted light that havebeen separated by such elements (e.g., generating images of lightemitted from the environment within respective different ranges ofwavelengths). Additionally or alternatively, the light received from theenvironment could be applied to a chromatically dispersive element(e.g., a prism, a diffraction grating, an Amici prism) to refract,reflect, or otherwise manipulate the light in a wavelength-selectivemanner, e.g., to spectrally separate the received light. Such separatedlight could then be imaged (e.g., by a one-dimensional ortwo-dimensional array of light-sensitive elements).

Such a chromatically dispersive optical element could be controllable,e.g., to control a direction and/or magnitude of thewavelength-dependent separation of different wavelengths of the receivedlight. The chromatically dispersive element could be controlled, overtime, to change the direction and/or magnitude of thewavelength-dependent separation of the received light, and multipleimages of the separated light could be taken corresponding to differentdirections and/or magnitudes of separation. Such multiple images couldthen be used to determine the location and color of fluorophores in anenvironment (e.g., via deconvolution of the images, based on informationabout the directions, magnitudes, or other information about thewavelength-specific separation of the light represented in each of theimages). The determined colors and locations of multiple fluorophorescould be used to identify probes in the environment, to determine thelocation and/or orientation of the probes, to determine the locationand/or concentration of analytes in the environment, or to determinesome other information about the probes, analytes with which the probesare configured to interact, and/or the environment from which the lightwas received.

In some examples, such a chromatically dispersive element could be aspatial light modulator (SLM) that includes a refractive layer disposedon a reflective layer. The refractive layer could be controllable (e.g.,electronically controllable) to have a refractive index that variesspatially across a surface of the SLM according to a controllablegradient (e.g., a substantially linear gradient). Further, thecontrollable refractive index of the refractive layer could bechromatically dispersive, i.e., dependent on the wavelength of lightrefracted by the refractive layer. A magnitude, direction, or otherproperty of the controllable gradient in the refractive index of the SLMcould be controlled according to an application, e.g., to control anangle of reflection of light incident on the SLM, to control a degree ofspectral dispersion of light reflected from the SLM (e.g., to control aspectral resolution at which an imager receiving the dispersed lightcould determine spectrographic information for the light reflected formthe SLM), or according to some other application.

In some examples, detecting the color and relative location offluorophores of the probes can include splitting light received from aparticular region of an environment (e.g., a region that is beingilluminated by a confocal microscope or otherwise confocally imaged)such that the light can be spatially imaged by a first light sensor andspectrally imaged by a second light sensor. This could include applyingthe received light to a partially silvered mirror, a polarizing mirror,or some other optical element(s) to split the light.

Spatially imaging a portion of the received light could includedetecting, using a plurality of light-sensitive elements (e.g., pixels)of the first light sensor, a respective plurality of time-varyingwaveforms of light (e.g., waveforms of the intensity of the light)received from respective different locations of the particular region ofthe environment. Such time-varying waveforms could be used to determinethe locations of different fluorophores in the region of theenvironment, e.g., by determining correlations between differenttime-varying waveforms. Such locations could be determined to aresolution beyond the diffraction limit of an optical system used toimage the received light. Light received from a particular fluorophore(e.g., from a particular quantum dot) can exhibit similar patterns ofintensity over time, such that there can be a higher correlation betweentime-varying waveforms of light received from locations proximate thelocation of the particular fluorophore.

Spectrally imaging a portion of the received light could includedetecting, using a plurality of light-sensitive elements (e.g., pixels)of the second light sensor, a respective plurality of time-varyingwaveforms of light received from the particular region of theenvironment at respective different wavelengths. This could includeapplying the portion of the received light to a chromatically dispersiveelement (e.g., a prism, a diffraction grating, an SLM as describedabove) such that different light-sensitive elements of the second lightsensor receive light having respective different wavelengths and/orwithin respective different ranges of wavelengths. Such time-varyingwaveforms could be used to determine the colors of differentfluorophores in the region of the environment, e.g., by determiningcorrelations between time-varying waveforms produced by the first lightsensor and time-varying waveforms produced by the second light sensor.Light received from a particular fluorophore (e.g., from a particularquantum dot) can exhibit similar patterns of intensity over time, suchthat there can be a higher correlation between time-varying waveforms oflight received from locations proximate the location of the particularfluorophore and time-varying waveforms of light at wavelengths that areemitted by the particular fluorophore (e.g., at one or more emissionwavelengths of the fluorophore).

Other configurations, modes and methods of operation, and otherembodiments are anticipated. For example, a target environment could beimaged, according to the methods described herein, when illuminated bylight at different wavelengths, e.g., wavelengths corresponding todifferent excitation wavelengths of respective different fluorophores ofone or more probes. Systems and/or methods described herein couldinclude additional microscopic or other imaging modalities and/oroptical systems or elements to improve the identification of probes asdescribed herein or other contents of portions of a target environmentaccording to an application. A system as described herein could includemultiple light sources, multiple spatial light modulators, multiplechromatically dispersive elements (e.g., SLMs, prisms, diffractiongratings), multiple light sensors (e.g., cameras, spectrometers),controllable apertures (e.g., according to a confocal imagingconfiguration), multiple micromirror devices, and/or additionalcomponents according to an application. Systems and methods describedherein could be used to identify and localize spatially and spectrallymultiplexed probes in a variety of different environments, e.g., invitro environments, in vivo environment, or ex vivo environments.Further, systems and methods as described herein could be configured oroperated according to and/or in combination with a variety of differentmicroscopic or other imaging techniques, e.g., stimulated emissiondepletion, ground state depletion, saturated structured illuminationmicroscopy, 4 pi imaging, photobleaching, or other methods ortechniques.

It should be understood that the above embodiments, and otherembodiments described herein, are provided for explanatory purposes, andare not intended to be limiting.

II. SPATIALLY AND SPECTRALLY MULTIPLEXED PROBES

As noted above, a number of different distinguishable fluorophores canbe assembled into probes such that the ordering, relative locations,colors, or other properties of the fluorophores can be detected and usedto identify different probes. By using multiple fluorophores (e.g.,quantum dots, fluorescent proteins, or Raman dyes, or other materialsthat can absorb and re-emit light) and controlling the number, ordering,and/or relative spacing of the fluorophores within each probe, thenumber of identifiable probes can be much greater than the number ofdistinguishable fluorophores used to produce the probes. Thefluorophores could be distinguishable by differing with respect to anexcitation spectrum, an excitation wavelength, an emission spectrum, anemission wavelength, a Raman spectrum, a fluorescence lifetime, or someother properties that may differ between different types of fluorophore.

Such probes could include fluorophores arranged in a linear arrangement,a circular or elliptical arrangement (e.g., in a ring), a triangulararrangement, or according to some other geometry. In some examples, thefluorophores could be arranged according to a three-dimensional geometry(e.g., at the vertices of a tetrahedron or other shape), and detectingthe relative locations of the fluorophores in order to identify theprobes could include determining the relative locations of thefluorophores in three dimensions (e.g., by three dimensional confocalimaging, by imaging probe from multiple different directions).

FIG. 1 illustrates a number of example probes 100 a, 100 b, 100 c. Theexample probes include different types of fluorophores 110 a, 110 b, 110c that have respective different colors (e.g., respective differentemission wavelengths, respective different emission spectra) and thatare arranged on respective backbone structures 120 a, 120 b, 120 c. Eachprobe 100 a, 100 b, 100 c further includes a respective binding agent130 a, 130 b, 130 c configured to selectively interact with a respectiveanalyte (e.g., by reversibly or irreversibly binding to the analyteand/or to an element of the analyte). The first 100 a and second 100 bprobes include fluorophores that are arranged in a substantially lineararrangement, while the third probe 100 c includes fluorophores in asubstantially circular arrangement.

As shown in FIG. 1, the first 100 a and second 100 b probes include thesame number of fluorophores and number of each type of fluorophore(specifically, each includes one of each type of fluorophore). However,the ordering of the types of fluorophores on each probe is different,such that detecting the colors (or other identifying information, e.g.,excitation spectrum, blinking dynamics) and locations of thefluorophores of a probe can facilitate identification of the probe. Aprobe could include multiple instances of a particular type offluorophore (e.g., two instances of a blue quantum dot and one instanceof a red quantum dot), could include multiple instances of a single typeof fluorophore, and/or could include no instances of one or more typesof fluorophores used to create a set of different probes.

The number of identifiable probes that is able to be created using aparticular number of different types of fluorophores can be related tothe number of fluorophores on each type of probe, the arrangement of thefluorophores on each probe (e.g., a linear arrangement, a ringarrangement), whether different types of probe have respective differentnumbers of fluorophores, or other factors. Note that the number ofidentifiably different probes can be reduced by symmetries betweenprobes that are different but that may appear similar when imaged. Forexample, the third probe 100 c could be indistinguishable from anotherprobe that includes the same fluorophores, in the same order, butoriented at a different angle on the backbone 120 c relative to thebinding agent 130 c and/or relative to some other element(s) of theprobe 100 c.

The fluorophores 110 a, 110 b, 110 c could include a variety ofsubstances and/or structures configured to inelastically scatter, absorband fluorescently re-emit, or otherwise absorb and re-emit light. Thefluorophores could include quantum dots, fluorescent proteins,fluorescent small molecules, Raman dyes, plasmonic rods or otherplasmonic structures, or other substances or elements or combinations ofelements configured to absorb light and to responsively emit light(e.g., by fluorescent absorption and re-emission, by inelasticscattering, by plasmonic excitation and photonic emission) in a mannerthat is detectable and that is distinguishable between different typesof fluorophores. Quantum dots may be preferred, due to resistance tophotobleaching and broad excitation spectra, such that multipledifferent quantum dots (e.g., quantum dots that emit light at respectivedifferent wavelengths in response to excitation) can be excited byillumination at a single wavelength. The different types of fluorophoresmay differ with respect to a wavelength of light emitted by thefluorophores; for example, the different types of fluorophores could bedifferent types of quantum dots, each configured (e.g., by having arespective diameter, layer thickness, or composition) to emit light at arespective wavelength within a range of wavelengths, e.g., to emit lightat a wavelength between approximately 500 nanometers and approximately800 nanometers. Further, different types of fluorophore could operatevia different mechanisms; for example, a first fluorophore of a probecould be a quantum dot, and a second fluorophore of the probe could be afluorescent protein.

The backbone (e.g., 120 a, 120 b, 120 c) of probes as described hereincould be composed of a variety of materials or substances. For example,such backbones could be composed of single- or double-stranded DNA. Abackbone could be composed of a single length of DNA folded back onitself and/or multiple lengths of DNA connected together to increase therigidity of the backbone, e.g., by forming a DNA origami structure. Thiscould include connected specified portions of one or more strands of DNAtogether using complementary staple strands of DNA. The fluorophorescould be connected to such staple strands and/or to the DNA to which thestaple strands are configured to bind, e.g., via covalent bonding orsome other mechanism. The sequences of the staple strands and/or of theDNA strands to which the staple strands are configured to bind could bespecified to control a shape, size, rigidity, or geometry of thebackbone and/or to control the location of fluorophores on such a formedbackbone. Different probes (e.g., probes having different numbers,types, spacings, and/or orderings of fluorophores) could be formed byusing different staple strands and/or different sequences of base DNA towhich such staple strands are configured to bind.

The rigidity of the backbone, the overall size of the probes, thespacing of the fluorophores, and/or some other properties of the probescould be specified to facilitate identification of the probes, motion ofthe probes within an environment of interest, or to facilitate someother process related to the probes and/or to an environment ofinterest. For example, the probes could be formed to have a length, adiameter, or some other largest dimension that is less thanapproximately 1 micron, e.g., to facilitate motion of the probes withincells or other environments of interest and/or to minimize the effect ofsuch probes on biological or chemical processes within a cell or otherenvironment of interest. Correspondingly, the fluorophores of suchprobes could be separated in space (e.g., along the length of a linearbackbone, or along the circumference of a ring-shaped backbone) by adistance that is sufficiently large that the colors of adjacentfluorophores can be resolved, e.g., by distances greater thanapproximately 20 nanometers. Further, the fluorophores could beseparated in space by a sufficiently small distance such that a desirednumber of fluorophores can be located on the backbone, e.g., bydistances less than approximately 50 nanometers. Further, the backbonecould have a rigidity that is sufficiently great that the backbone issubstantially rigid across distances greater than the distances betweenthe fluorophores on the backbone, e.g., such that the order offluorophores along the backbone of the probe can still be determinedwhen the backbone is being deformed by thermal process or other forcesexpected to be present in an environment of interest.

As noted above, each type of probe (e.g., 100 a, 100 b, 100 c) could beconfigured to selectively bind with or otherwise selectively interactwith an analyte of interest, or to otherwise selectively interact withan environment of interest (e.g., to be selectively taken up viaendocytosis by a cell type of interest) to facilitate detection of someproperties of the environment of interest (e.g., to detect a location orconcentration of a protein, a DNA segment, a cytoskeletal element, orsome other analyte of interest). This could include the probes includingrespective different binding agents 130 a, 130 b, 130 c that areconfigured to selectively bind to or otherwise selectively interact withan analyte of interest. Such a binding agent could include a protein, anantibody, a receptor, a recognition protein, a DNA segment, an aptamer,an RNA segment, a small molecule, or some other element configured toselectively interact with and/or to be interacted with by an analyte ofinterest.

III. IDENTIFYING PROBES BY REPEATED IMAGING USING A SPATIAL LIGHTMODULATOR

A variety of methods could be used in order to locate and identify, inan environment of interest, spatially and spectrally multiplexed probesas described elsewhere herein. Such methods can include detecting thelocations and colors of fluorophores in the environment and using suchdetermined information to determine the identity or other information(e.g., location, orientation) of probes in the environment by matchingdetected patterns of fluorophore types (e.g., orders of colors oflinearly, circularly, or otherwise arranged patterns fluorophores)within the environment to known patterns of fluorophores thatcorresponds to the identities of potential probes in the environment.Detecting the color of a fluorophore could include determining anemission spectrum, a characteristic wavelength, or some othercharacteristic(s) of the spectral content of light emitted from afluorophore.

Detection of the locations and colors of fluorophores can includeapplying light received from an environment of interest (e.g., lightemitted from fluorophores in the environment in response toillumination) to a chromatically dispersive element (e.g., a prism, adiffraction grating, or some other element(s) configured to reflect,refract, and/or diffract light in a wavelength-specific manner) andimaging light that is spectrally dispersed due to interaction with thechromatically dispersive element. Light that is spectrally dispersed mayexhibit a change in an angle of propagation of the light, a direction ofpolarization of light, or a change in some other properties of the lightthat is related to the wavelength of the light. This could includechanging, in a wavelength-dependent way, an angle of propagation oflight received from an environment such that redder wavelengths of suchspectrally dispersed light are shifted, relative to light-sensitiveelements (e.g., pixels) of a light sensor (e.g., a camera), in a firstdirection by a first amount while bluer wavelengths are shifter in thefirst direction by a lesser amount and/or are shifted opposite the firstdirection. An image generated from such spectrally dispersed light willbe distorted, relative to a non-spectrally dispersed image of theenvironment, in a manner that is related to the direction, magnitude, orother property of the spectral dispersion caused by the chromaticallydispersive element. By controlling such a direction, magnitude, and/orother property of the spectral dispersion of the light during differentperiods of time and imaging the light so dispersed during the differentperiods of time, the color and location of fluorophores in theenvironment can be determined.

In some examples, a 2-dimensional array of light sensitive elements of alight sensor (e.g., a 2-dimensional array of pixels of a camera) couldbe used to image such spectrally dispersed light. In such examples,wherein light is received from a plurality of regions of a target (e.g.,as in bright-field microscopy), each light-sensitive element of such a2-dimensional array could receive light of a variety of differentwavelengths from a variety of respective different locations of thetarget. A correspondence between an individual light-sensitive elementof such an array to light of a range of locations of a target at a rangeof corresponding wavelengths could be determined (e.g., by modeling orsimulation of elements of such an imaging system, by empirical testingof such a system using one or more calibration targets having respectiveknown patterns of spectrographic properties) and such a correspondencecould be used to determine spectrographic information (e.g., a color)for one or more locations of an imaged target based on a number ofimages of the target taken while operating a chromatically dispersiveelement according to a respective number of different directions and/ormagnitudes of spectral dispersion of the received light (e.g., via aprocess of deconvolution). Such information could be used to determinethe locations and colors of fluorophores in the target.

In an example, the chromatically dispersive element could include aprism, diffraction grating, or other element(s) that is mechanicallyactuated, e.g., to control a direction of the spectral dispersion oflight applied to the prism. In another example, the chromaticallydispersive element could include a spatial light modulator (SLM)composed of a chromatically dispersive refractive layer disposed on areflective layer. Controlling a direction and magnitude of a gradient ofthe refractive index of the refractive layer across the SLM couldprovide control of a direction, magnitude, or other properties of thespectral dispersion of the imaged light received from the target.

FIG. 2A illustrates a target 200 a. Within the target 200 a is a probe205 that includes, in order, a green fluorophore, Ga, a red fluorophore,Ra, and a blue fluorophore, Ba, disposed on a substantially linearbackbone. Thus, spectrographic properties (e.g., colors) of locations ofthe target 200 a are such that red light is emitted from the location ofRa in response to illumination, green light is emitted from the locationof Ga in response to illumination, and blue light is emitted from thelocation of Ba in response to illumination. The target 200 a could beimaged by an imaging system as described elsewhere herein.

FIG. 2B illustrates a portion of a first image 200 b of the target 200a. This first image 200 b is taken of light received from the targetthat has been spectrally dispersed by a chromatically dispersive elementduring a first period of time. In this example, the chromaticallydispersive element includes an SLM that includes a chromaticallydispersive refractive layer disposed on a reflective layer, wherein adirection and magnitude of a gradient of the refractive index of therefractive layer across the SLM is controllable. The first image 200 bincludes illuminated regions Rb, Gb, and Bb due to illumination ofcorresponding regions of a light sensor by spectrally dispersed lightfrom the red, green, and blue fluorophores (Ra, Ga, and Ba),respectively, of the probe 205. The SLM is operated during the firstperiod of time such that its refractive layer has a refractive indexthat varies spatially across the SLM according to a gradient in a firstdirection (indicated by the arrow 210 b) such that light of differentwavelengths is dispersed in the first direction 210 b when imaged by alight sensor (e.g., as in the first image 200 b). Such dispersionaffects imaging of the dispersed light during the first period of timeby shifting light at longer wavelengths farther in the direction of thearrow within the first image 200 b; as a result, the first image 200 bof the target 200 a includes illuminated regions Rb, Gb, and Bb arrangedas shown.

An imaging system as described elsewhere herein could be operated inthis way during a plurality of further periods of time to generate afurther plurality of respective images of light received from the targetand dispersed by the SLM (or by some other controllable chromaticallydispersive element(s)). The SLM could be operated during such furtherperiods of time such that its refractive layer has a refractive indexthat varies spatially across the SLM according to respective gradientsin respective further directions and/or having respective furthermagnitudes or according to some other set of respective patterns. FIGS.2C and 2D illustrate portions of a second image 200 c and a third image200 d, respectively, of the target 200 a. The second image 200 c andthird image 200 d are taken of light received from the target that hasbeen spectrally dispersed by the SLM during respective second and thirdperiods of time. The second image 200 c and third image 200 d includerespective sets of illuminated regions Rc, Gc, and Bc and Rd, Gd, and Bddue to illumination of corresponding regions of the light sensor bydispersed light from the red, green, and blue fluorophores (Ra, Ga, andBa), respectively, of the probe 205.

The SLM is operated during the second and third periods of time suchthat its refractive layer has a refractive index that varies spatiallyacross the SLM according to a gradient in a second direction and a thirddirection, respectively (indicated by the arrows 210 c, 210 d,respectively) such that light of different wavelengths is dispersed inthe second direction 210 c and third direction 210 d when imaged duringthe second and third periods of time by the light sensor (e.g., as inthe second 200 c and third 200 d images). Such dispersion affectsimaging of the dispersed light during the second and third periods oftime by shifting light at longer wavelengths farther in the direction ofrespective arrows within the second 200 c and third 200 d images. As aresult, the second image 200 c of the target 200 a includes illuminatedregions Rc, Gc, and Bc and the third image 200 d of the target 200 aincludes illuminated regions Rd, Gd, and Bd arranged as shown.

Such multiple images of the target 200 a, taken from light dispersed inrespective multiple ways by the SLM operated according to respectivemultiple configurations of refractive index (e.g., according togradients having respective different directions and/or magnitudes)could be used to determine spectrographic information (e.g., colors) forone or more locations (e.g., particular region Pa) of the target 200 aand/or to determine the location of fluorophores (g., Ra, Ga, Ba) orother light-emitting contents of the target 200 a. In some examples,such information could be determined for a plurality of regions acrossthe target 200 a allowing, e.g., hyperspectral imaging of the target 200a. A plurality of such images, in combination with a model or otheralgorithm describing the effects of the plurality of patterns ofrefractive index of the SLM and/or the effects of such configurations todisperse light received from the target 200 a during the periods of timecorresponding to the plurality of images. Such a determination couldinclude a process of deconvolution or some other computational process.

In an illustrative example, spectrographic information about theparticular region Pa of the target 200 a, corresponding to the locationof the red fluorophore, Ra, could be determined based on the amplitudeor other detected information about light detected at regions of thelight sensor (e.g., by one or more light-sensitive elements or pixels ofthe light sensor) corresponding, according to the location of theparticular region Pa and the dispersive effects of the SLM during theplurality of periods of time corresponding to the plurality of images.

For example, an amplitude of red light emitted from Pa, where the redfluorophore is located, in response to illumination by the imagingsystem could be determined based on a linear combination or otherfunction of the light detected at points Prb, Prc, and Prd in the first200 a, second 200 b, and third 200 c images of the target. Similarly, anamplitude of green light emitted from Pa in response to illumination bythe imaging system could be determined based on a linear combination orother function of the light detected at points Pgb, Pgc, and Pgd in thefirst 200 a, second 200 b, and third 200 c images of the target and anamplitude of blue light emitted from Pa in response to illumination bythe imaging system could be determined based on a linear combination orother function of the light detected at points Pbb, Pbc, and Pbd in thefirst 200 a, second 200 b, and third 200 c images of the target. Theamount of green and blue light emitted from Pa would be less than theamount of red light emitted from Pa because the red fluorophore emitsmore light, in response to being illuminated, at red wavelengths than atblue or green wavelengths. An intensity of light at a variety ofdifferent wavelength that is emitted from Pa could be determined, basedon the light detected at respective different points in respectivedifferent images, and used to determine the color of a fluorophore inthe environment 200 a.

The location of such corresponding locations (e.g., Prb, Prc, Prd, Pgb,Pgc, Pgd, Pbb, Pbc, Pbd) could be determined based on a model of animaging system (e.g., based on the magnitude and direction of a gradientof refractive index of the refractive layer across the SLM) and/or on anempirical measurement of the properties of the imaging system (e.g.,based on a set of images of a calibration target having knownspectrographic information/content or some other calibration informationor procedure). Note that the colors (red, green, and blue) and operationof the SLM to disperse light in the illustrated different directions areintended as non-limiting examples; different wavelengths and/or rangesof wavelengths of spectrographic information could be determined forlocations of a target, e.g., locations of fluorophores in a target.Further, an SLM could be operated to have a pattern of refractive indexaccording to gradients having respective different directions,magnitudes, or according to some other set of patterns of refractiveindex.

An SLM as described herein and used to provide hyperspectral imagingand/or the determination of spectrographic data (e.g., a color) for oneor more locations of a target (e.g., for locations of fluorophores ofprobes in a target) has one or more chromatically dispersive propertiesthat are electronically (or otherwise) controllable and that allow theSLM to spectrally disperse light presented to the SLM. A chromaticallydispersive property of an object or material is an optical property thathas a dependence on the wavelength of light interacting with the objector material. For example, certain glasses have chromatically dispersiverefractive indexes in that the refractive indexes of the glasses aredifferent for different wavelengths of light. In another example,certain diffraction gratings have different effective absorbances and/orangles of reflection for different wavelengths of light. Thus, suchobjects or materials having chromatically dispersive properties can beused to spectrally disperse light, i.e., to interact with light appliedto the object or material in a wavelength-dependent manner such thatlight emitted from the object or material (e.g., reflected from,absorbed by, transmitted through, optically rotated by) has one or moreproperties (e.g., an angle, an amplitude, an orientation ofpolarization) that are wavelength-dependent that were substantially notwavelength-dependent in the applied light. As an example, a prism (e.g.,a triangular prism) composed of a glass having a chromaticallydispersive refractive index could interact with a beam of white light(e.g., a beam containing light at a variety of amplitudes across thevisible spectrum) such that light emitted from the prism at a variousvisible wavelengths is emitted at respective different angles (e.g., asa ‘rainbow’).

An example of such an electronically-controlled chromatically dispersiveelement is illustrated in cross-section in FIG. 3A. FIG. 3A illustratesthe configuration of a spatial light modulator (SLM) 300 that includes areflective layer 320 (composed of, e.g., aluminum, silver, or some othermaterial that is reflective to light within a range of wavelengths ofinterest) disposed beneath a refractive layer 310. A substantiallytransparent first electrode 340 (composed, e.g., of indium-tin-oxide(ITO) or some other material that is electrically conductive andsubstantially transparent to light within a range of wavelengths ofinterest) is located on the refractive layer 310 opposite from thereflective layer 320. Light directed toward the SLM 300 could betransmitted through the first electrode 340, refracted by the refractivelayer 310, reflected by the reflective layer 320, refracted again by therefractive layer 310, and transmitted away from the SLM 300 through thefirst electrode 340. The SLM 300 additionally includes a dielectriclayer 350 and a plurality of further electrodes 330 (including second335 a, third 335 b, and fourth 335 c electrodes) disposed beneath thereflective layer 320. A controller 360 is configured to control voltagesbetween the first electrode 340 and each of the further electrodes 330.Note that the reflective layer 320 and dielectric layer 350 areillustrated as distinct structures of the SLM 300, but in practice couldbe the same structure (e.g., the dielectric layer 350 could be composedof a reflective material such that the reflective layer 320 is simplythe surface of the dielectric layer 350, the reflective layer 320 couldcomprise a polished or otherwise formed or treated surface of thedielectric layer 350 such that the reflective layer 320 is reflective).

The refractive layer 310 is composed of a material (e.g., a liquidcrystal) that is chromatically dispersive with respect to its refractiveindex. That is, the refractive index of the refractive layer 310 dependson the wavelength of light refracted by the refractive layer 310. Insome examples, the refractive index of the refractive layer 310 couldvary substantially linearly with wavelength for wavelengths within aspecified range of wavelengths (e.g., visible wavelengths, a range ofwavelengths including emission wavelengths of two or more fluorophores).Further, the refractive index of the refractive layer 310 can becontrolled electronically by applying a controlled electric field to therefractive layer 310, e.g., by applying a voltage between the firstelectrode 340 and one or more of the further electrodes 330. Therefractive index of the refractive layer 310 could be related to alinear or nonlinear function of a DC voltage, an amplitude, frequency,duty cycle, pulse width, or other property of an AC voltage, or someother property of voltage applied between the first electrode 340 andone or more of the further electrodes 330. Further, the refractive indexof individual regions or cells of the refractive layer 310 could becontrolled independently or semi-independently by applying differentvoltages, voltage waveforms, or other different electronic signalsbetween the first electrode 340 and one or more of the furtherelectrodes 330 corresponding to the individual regions or cells of therefractive layer 310. For example, the refractive index of first 315 a,second 315 b, and third 315 c regions of the refractive layer 310 couldbe controlled by controlling a voltage or voltage waveform appliedbetween the first electrode 340 and the first 335 a, second 335 b, andthird 335 c further electrodes, respectively.

Note that the SLM 300 is illustrated in cross-section in FIG. 3A andthus shows only a single row of regions (e.g., 315 a-c) andcorresponding electrodes (e.g., 335 a-c) of the SLM 300. The SLM 300could include a regular, two-dimensional array of such regions. Such anarray could include a rectangular, square, hexagonal, or other repeatingor non-repeating array of such regions and electrodes. Alternatively, anSLM could be configured to have electrodes and corresponding cells orother regions of a refractive layer according to some other pattern orapplication, e.g., a repeating pattern of linear electrodes (e.g., a1-dimensional array of regions across the surface of the SLM). Thevoltages, voltage waveforms, or other electronic signals applied to theelectrodes could be controlled such that the refractive index of therefractive layer varies across the surface of the SLM according to aspecified pattern, e.g., according to a locally or globallysubstantially linear or nonlinear gradient. Such a local or globalgradient could have a specified magnitude, a specified direction, orsome other specified properties. Further, such specified patterns (e.g.,gradients) could be changed over time according to some application. Forexample, light could be received from a target, reflected from such anSLM, and imaged by a light sensor, camera, or other imaging element toallow image capture of light received from a target during a pluralityof periods of time when operating the SLM according to respectivedifferent patterns (e.g., gradients having respective specifiedmagnitudes and directions) to spectrally disperse the imaged light in aplurality of respective ways, allowing determination of spectrographicinformation for regions of the target based on the plurality of images,e.g., via a process of deconvolution.

FIG. 3B illustrates a variety of functions describing the dependence ofthe refractive index of regions of the refractive layer 310 (thevertical axis, ‘RI’) on the wavelength of refracted light (thehorizontal axis, ‘WAVELENGTH’) when composed of different materialsand/or when exposed to different electrical fields (e.g., when aspecified voltage or voltage waveform is applied between the firstelectrode 340 and one of further electrodes 330 corresponding to aregion of the SLM 300). ‘B’, ‘G’, and ‘R’ indicate the wavelengths ofblue, green, and red light, respectively.

Functions X, Y, and Z illustrate the wavelength-dependent refractiveindex of a first refractive layer material composition. The firstrefractive layer material composition has a refractive index that variessubstantially linearly across the illustrated range of wavelengths.Functions X, Y, and Z illustrate the refractive index of a region of thefirst refractive layer material composition as an applied electronicsignal is varied (e.g., X, Y, and Z are the refractive index of theregion as a voltage between electrodes opposite the cell is increased).X, Y, and Z show increasing overall refractive index as well as adecreasing slope of dependence between the refractive index andwavelength. Similarly, functions V and W illustrate thewavelength-dependent refractive index of a second refractive layermaterial composition; V and W illustrate the refractive index of aregion of the second refractive layer material composition as an appliedelectronic signal is varied.

Note that the illustrated functions are intended to illustrateconfigurations and operations of embodiments described herein, and notto limit the embodiments described herein or to describe any particularrefractive layer material composition or dependence of opticalproperties thereof on electronic signals. A refractive index at one ormore wavelengths, a slope and/or offset of the refractive index across arange of wavelengths, a nonlinearity of the relationship between therefractive index and wavelength, or some other property of therefractive index of material included in a refractive layer of an SLM asdescribed herein could change linearly or nonlinearly with one or moreproperties of an applied electrical signal (e.g., an electric fieldmagnitude, an electric field direction, an applied current magnitude, anapplied current direction, a frequency, duty cycle, pulse width, orother property of an applied electrical signal).

FIG. 3C illustrates the use of an SLM 301 configured similarly to SLM300 and having a refractive layer composed of the first materialcomposition. The SLM 301 is operated such that the refractive layer hasa substantially linear gradient of refractive index between thelocations indicated by ‘X’ and ‘Y’ and such that the locations indicatedby ‘X’ and ‘Y’ have wavelength-dependent refractive indexescorresponding to the functions ‘X’ and ‘Y’, respectively (e.g., bycontrolling electrodes of regions proximate to ‘X’ and ‘Y’ according tocorresponding voltages or voltage waveforms and controlling one or moreregions located between ‘X’ and ‘Y’ according to some intermediatevoltages). Incoming light 380 c includes light at wavelengthscorresponding to the ‘R’, ‘G’, and ‘B’ indications in FIG. 3B. Theincoming light 380 c is reflected and refracted by the SLM 301 andemitted as reflected light 390 c. Due to the wavelength-dependence ofthe refractive index of the refractive layer of the SLM 301, reflectedlight 390 c is spectrally dispersed (illustrated as separate ‘R’, ‘G’,and ‘B’ rays of light). The angle of each ray of the reflected light 390c could be related to the thickness of the refractive layer of the SLM301 and the pattern of change of the refractive index of the refractivelayer for each ray across the refractive layer. For example, the angleof the ‘B’ ray could be related to a magnitude and/or angle of agradient of the refractive index of the refractive layer for light atwavelength ‘B’ across the SLM 301 in the area proximate the intersectionof the SLM 301 and the incoming light 380 c.

An amount of spectral dispersion of light reflected by an SLM could beincreased by increasing a magnitude of a gradient or other rate ofchange in a pattern of the refractive index of the refractive layer.Such an increase in spectral dispersion could allow spectrographicinformation for a received light to be determined with a higher spectralresolution, e.g., by causing light of two different wavelengths to bedetected by light-sensitive elements (e.g., pixels) of a light sensorthat are farther apart by increasing an angle between rays of dispersedlight at the two different wavelengths.

Note that the described regular array of electrodes disposed as part ofan SLM to allow the electronic control of the refractive index ofrespective cells or other specified regions of a refractive layer (orother refractive element(s)) of the SLM is intended as one exampleembodiment of an SLM having a refractive layer having a refractive indexthat can be electronically controlled to vary across the refractivelayer according to a controllable gradient having at least one of aspecified direction or magnitude. Alternative embodiments couldelectronically control one or more lasers or other light sources tooptically control the refractive index of a refractive element of anSLM. Other configurations and operations of an SLM as described hereinare anticipated. Further, an SLM could be operated in a transmissivemode, i.e., could lack a reflective layer. In such examples, a beam oflight (e.g., a beam of light received from an illuminated target) couldbe spectrally dispersed by the SLM by being transmitted through arefractive layer of the SLM that has a pattern of refractive index thatcan be electronically controlled. In some examples, an SLM could act toprovide electronically controlled spectral dispersion of a beam of lightby controlling a pattern of reflective and absorptive elements on asurface and/or within a volume of the SLM to provide a diffractiongrating having one or more properties (e.g., a grating spacing, agrating width, a grating orientation) that can be electronicallycontrolled to control one or more properties of spectrally dispersedlight reflected from and/or transmitted through the SLM in response toreceiving light from a target.

In order to produce images of a target, as described above, that can beused to identify probes in the target by determining the locations andcolors of fluorophores of the probes in the target, an SLM (e.g., 200)or other chromatically dispersive element as described elsewhere hereincould be incorporated into an imaging system that includes additionalelements, e.g., sources of illumination, optical systems, apertures,light sensors, or other elements configured to illuminate a target, toapply light responsively emitted from the target (e.g., fromfluorophores of probes of the target) to the SLM and then to apply suchlight that has interacted with (e.g., by spectrally dispersed by) theSLM to a light sensor to be imaged. Such illumination and/or receptionof light can be of/from a wide area of the target (e.g., bright-fieldmicroscopy) or of/from some specified region of the target (e.g., aplurality of specified small volumes of the target, as in confocalmicroscopy). Spectrographic information could be detected/determined forone or more regions of the target by illuminating the target withmultiple lights having multiple respective spectrographic properties(e.g., containing light at multiple respective wavelengths) and/or bydetecting a wavelength-dependence of the amplitude or other propertiesof the received light (e.g., by detecting the amplitude of the receivedlight within multiple ranges of wavelengths).

FIG. 4 illustrates in cross-section elements of an example imagingsystem 400 configured to image a target 405. The system 400 includes alight source 420 (e.g., a laser), a light sensor 430 (illustrated as aplane of light-sensitive elements located on a focal plane 437 of thelight sensor 430), a micromirror device (MD) 450 (that includes aplurality of electronically actuated micromirrors located on a focalplane 457), a spatial light modulator (SLM) 410, and an optical system(including an objective 441, first 443 and second 444 relay lenses, adichroic mirror 445, and an optical sink 425) configured to direct lightto and from the target 405 and between the elements of the system 400.The system 400 additionally includes a stage 460 to which the target 405is mounted. Note that the MD 450 and light sensor 430 comprisetwo-dimensional arrays of micromirrors and light-sensitive elements,respectively. Further, note that the optical system (e.g., 441, 443,444, 445) and SLM 410 are configured to direct light between the target,405, MD 450, and light sensor 430 such that locations on the focalsurfaces 457, 437 of the MD 450 and light sensor 430 correspond torespective locations on a focal surface 407 that passes through thetarget 405.

The system 400 illuminates a specified region 409 on the focal surface407 in the target 405 by emitting a first illumination 421 from thelight source 420 and reflecting the first illumination 421 from thedichroic mirror 445 toward the MD 450. A selected set of at least onemirror 451 of the MD 450 (illustrated in FIG. 4 as a single mirror) thathas a location on a focal surface 457 of the MD 450 corresponding to thespecified region 409 is controlled to reflect the first illumination 421toward the target 405 as in-focus illumination 422 via the objective441. Other mirrors 453 of the MD 450 are controlled to reflect theremainder of the first illumination 421 as waste illumination 423 towardthe optical sink 425 to be absorbed. As illustrated, a single mirror(451) is controlled to illuminate (and to receive light from) acorresponding region 409 of the target 405; however, additional mirrors(e.g., selected from other mirrors 453) could be operatedsimultaneously, sequentially, or according to some other scheme toilluminate (and to receive light from) corresponding additional regionsof the target 405.

The system 400 receives light (including conjugate light 432) emittedfrom the specified region 409 in response to illumination via theobjective 441. The conjugate light 432 arrives, in-focus, at theselected mirror 451 and is reflected (through the dichroic mirror 445)toward the SLM 410. The first relay lens 443 (and/or some other opticalelements of the system 400) collimates the received light and presentsthe substantially collimated light to the SLM 410. The SLM 400 reflectsthe conjugate light 432 as spectrally dispersed light 433 toward thesecond relay lens 444 that is configured to present the spectrallydispersed light 433 in-focus to a specified region 431 on a focalsurface 437 of the light sensor 430 corresponding to the specifiedregion 409 (e.g., to a region of the light sensor having one or morelight-sensitive elements and/or pixels of the light sensor 430). The SLM410 is configured and/or operated such that the spectrally dispersedlight 433 is spectrally dispersed relative to the conjugate light 432 ina controlled manner such that spectrographic information of theparticular region 409 and/or of the conjugate light 432 (e.g., a colorand/or location of one or more fluorophores of one or more probeslocated at the particular region 409) can be detected or determined. Insome examples, the spectrally dispersed light 433 is spectrallydispersed in a manner related to an electronically controlled direction,magnitude, and/or some other property of a spatial gradient in therefractive index of a layer of the SLM 410.

Note that the system 400 and elements thereof shown in FIG. 4 areintended as a non-limiting example of systems and methods as describedelsewhere herein for generating hyperspectral or otherwisespectrographic images of a target (e.g., 405) in order to, e.g.,identify probes within a target by determining locations and colors offluorophores of probes in the target. Imaging systems could include moreor fewer elements, and could image a target according to similar ordifferent methods. As shown, the system 400 can be operated to image thetarget 405 confocally; i.e., to illuminate a specified region of thetarget 409 in-focus and to receive light responsively emitted from thespecified region 409 in-focus using the micromirror device 450 (e.g., tocontrol the spatial pattern of light emitted toward and received fromthe target 405). Illumination could be delivered to the target 405 andlight received from the target 405 in different ways and usingdifferently configured elements (e.g., different optics). The target 405could be illuminated along an optical path separate from the opticalpath used to receive light responsively emitted from the target 405. Forexample, illumination could be transmitted through a target before beingreceived to image the target. Particular regions of the target 405 couldbe illuminated, and light received from such regions, by steering a beamof illumination using one or more controllable mirrors, lenses,diffraction gratings, or other actuated optical elements.

An SLM (e.g., 410) as described herein could be configured and operatedas part of a variety of different imaging systems (e.g., bright-fieldmicroscopes, 4-pi microscopes, confocal microscopes, fluorescencemicroscopes, structured illumination microscopes, dark fieldmicroscopes, phase contrast microscopes) to provide controlled spectraldispersion of light for a variety of applications (e.g., to allowhyperspectral or otherwise spectrographic imaging of a target in orderto identify spatially and spectrally multiplexed probes within thetarget). For example, an SLM as described herein could be inserted intothe path of light received by some other variety of microscope or imager(e.g., a bright-field microscope). The SLM could be operated to have aplurality of different specified magnitudes and/or directions ofrefractive index gradient across the SLM during a respective pluralityof periods of time, and such an imager could be configured to generate aplurality of images of the received light reflected from the SLM duringthe plurality of periods of time. In such examples, spectrographicinformation about a particular portion of a target (e.g., a target fromwhich the received light is received) could be determined based on aplurality of detected amplitudes (or other properties of light) ofpixels across the plurality of images according to a model (e.g., ablack-box model fitted to calibration data for the imager) or otherdescription of the relationship between the detected amplitudes andspectrographic properties (e.g., colors) of regions of the target (e.g.,regions that include one or more fluorophores of one or more probes)depending on the configuration of the SLM (e.g., via a process ofdeconvolution performed on the plurality of images and based on awavelength-dependent point-spread function determined for the imager).Further, an SLM as described herein could be used to control a directionand/or spectral content of a beam of illumination, e.g., to effect atunable light source in combination with a source of broad-spectrumlight and, e.g., an aperture.

The light source 420 could include a variety of light-emitting elementsconfigured to produce illumination 421 having one or more specifiedproperties (e.g., specified wavelength(s)). This could include lasers,light-emitting diodes (LEDs), or other substantially monochromatic lightsources. Additionally or alternatively, the light source 420 couldinclude a light emitting element that emits light across a wider rangeof wavelengths (e.g., an arc lamp). In some examples, thisnon-monochromatic light could be emitted through one or more filters(e.g., filters including one or more Bragg reflectors, prisms,diffraction gratings, slit apertures, monochromators) configured to onlyallow the transmission of light within a narrow range of wavelengths. Insome examples, the light source 420 could be configured to emit light ata specified wavelength or having some other specified property to excitea fluorophore (e.g., to excite a fluorescent protein, to excite one ormore types of quantum dots) in the target 405 or to otherwiseselectively interact with (e.g., excite, quench, photobleach) one ormore elements of the target 420. For example, the illumination 421 couldinclude light at substantially one wavelength (i.e., could contain lightof wavelengths within a specified narrow range of wavelengths)corresponding to an excitation wavelength of a fluorophore (e.g., agreen fluorescent protein, a dsRED protein) in the target 405.

In some examples, the light source 420 could include a tunable laser orsome other light-emitting element(s) controllable to emit light at anyof a plurality of different wavelengths (e.g., wavelengths rangingbetween approximately 400 nanometers and approximately 2.5 micrometers).Such a tunable laser could include an excimer laser, a dye laser, a CO₂laser, a free-electron laser, or some other laser element configured toemit light at a plurality of different, controllable wavelengths. Insome examples, the wavelength of the light emitted by such a tunablelaser could be controlled by controlling a geometry or size of one ormore elements (e.g., a reflector, a resonating cavity) of the tunablelaser. In some examples, a Bragg reflector or other element of the lightsource 420 (e.g., of a tunable laser) could be rotated or otherwiseactuated to control the wavelength of light emitted by the light source420. In some embodiments, the light source 420 could include a pluralityof lasers or other sources of substantially monochromatic lightconfigured to emit light at wavelengths corresponding to respectivedifferent wavelengths (e.g., excitation wavelengths of respectivefluorophores in the target 405), and operation of the light source 420to emit light of a particular wavelength could include operating thecorresponding laser of the light source 420 to emit light at thecontrolled wavelength. Other configurations and operations of a lightsource 420 are anticipated.

The light sensor 430 could include a plurality of light-sensitiveelements disposed on the focal surface 437. The light-sensitive elementscould be configured to detect the intensity or other properties of lightreceived by the light sensor 430 across a broad range of wavelengths(e.g., across a range of wavelengths of light that can be emitted byelements of the target 405, e.g., a range that includes emissionwavelengths of one or more quantum dots, fluorescent proteins, or otherfluorophores in the target 405). That is, the light sensor 430 could beconfigured to act as broadband monochrome light sensor, receiving lightfrom the target 405 (via, e.g., the SLM 410, MD 450, and optical system)during a plurality of periods of time and outputting a respectiveplurality of images related to the absorption, fluorescent re-emission,or other interactions of the target 405 with light (e.g., light of acorresponding plurality of wavelengths) emitted by the light source 420during a the respective plurality of periods of time. This could includethe light sensor 430 containing a regular two-dimensional (or otherwisearranged) array of light sensitive elements (e.g., photodiodes,phototransistors, pixels of a charge-coupled device (CCD), active pixelsensors) disposed on the focal surface 437 configured such that theoutput of an individual light sensitive element is related to theintensity of the light received by the light sensor 430 from aparticular direction and at a particular wavelength (corresponding to aparticular portion of the target 405 and the configuration of the SLM410 and/or MD 450).

Note that the configuration and/or operation of the system 400 toilluminate and to receive light from a specified region 409 on a focalsurface 407 of the target 405 is intended as a non-limiting example.Alternatively, a larger and/or differently-shaped region of the target(e.g., a line within the target; substantially the entire target and/orthe entire target within a field of view of the imaging system) could beilluminated by operating the mirrors 451, 453 of the MD 450 according toa different set of controlled angles than those illustrated. Forexample, a plurality of spatially separated regions proximate to thefocal surface 407 of the target 405 could be illuminated and imagedsimultaneously by controlling a corresponding plurality of spatiallyseparated mirrors of the MD 450 to reflect the first illumination 421toward the plurality of the regions of the target 405. The mirrors 451,453 of the MD 450 could be controlled according to some other pattern,e.g., to approximate some other coded aperture on the focal surface 457of the MD 450. Further, the light source 420 could emit illumination ata controllable wavelength (e.g., illumination that is substantiallymonochromatic, but having a wavelength that can be altered by operationof the light source) and spectrographic information could be determinedfor regions of the target 405 based on images of the target 405generated when the target 405 is illuminated by different wavelengths oflight (e.g., to generate a corresponding plurality of emission spectrafor the region corresponding to the different wavelengths ofillumination).

Further, note that the location of the focal surface 407 within thetarget 405 could be controlled (e.g., to allow imaging of elements ofthe target 405 at different depths within the target 405). In someexamples, the stage 460 could be actuated relative to other elements ofthe system 400 (e.g., relative to the objective 441) such that alocation of the target 405 in one or more dimensions could becontrolled. For example, the stage 460 could be actuated in a directionparallel to the direction of the conjugate illumination 432 (i.e., inthe vertical direction of FIG. 4) such that the location (e.g., thedepth) of the focal surface 407 within the target 405 could becontrolled. In such an example, a plurality of images and/orspectrographic information could be detected/determined of the target405 when the focal surface 407 is controlled to be at variety ofrespective locations (e.g., depths) within the target 405, allowing a3-dimensional image of the target 405 to be generated from the pluralityof images and/or spectrographic information. In some examples, thelocation of the particular region 409 on the focal surface 407 withinthe target 405 could be controlled by actuating the stage 460 to controlthe location of the target 405 relative to the system. Actuation of thestage 460 could include one or more piezo elements, servo motors, linearactuators, galvanometers, or other actuators configured to control thelocation of the stage 460 (and a target 405 mounted on the stage 460)relative to element(s) (e.g., 441) of the system 400.

The imaging system 400 (or other example imaging and/or microscopysystems described herein) could include additional elements orcomponents (not shown). The imaging system 400 could include one or morecontrollers configured to operate the SLM 410, light source 420, lightsensor 430, MD 450, actuator(s) configured to control the location ofthe stage 460, and/or other elements of the imaging system 400. Theimaging system 400 could include communications devices (wirelessradios, wired interfaces) configured to transmit/receive informationto/from other systems (e.g., servers, other imaging devices,experimental systems, sample perfusion pumps, optogenetic or otherstimulators) to enable functions and applications of the imaging system400. For example, the imaging system 400 could include an interfaceconfigured to present images of the target 405 generated by the imagingsystem 400 and/or images of the location, distribution, concentration,or other information about identified probes within the target 405. Theimaging system 400 could include an interface configured to presentinformation about the imaging system 400 to a user and/or to allow theuser to operate the imaging system 400 (e.g., to set a spectrographicresolution, to set a spatial resolution, to set a temporalresolution/imaging sample rate, to set an operational mode (e.g.,conjugate or non-conjugate confocal imaging, bright-field imaging,stimulated emission depletion (STED) imaging), so set a maximum emittedillumination power, to set a range of wavelengths of interest).

Additionally or alternatively, the imaging system 400 (or other exampleimaging systems described herein) could be configured to communicatewith another system (e.g., a cellphone, a tablet, a computer, a remoteserver) and to present a user interface using the remote system. In someexamples, the imaging system 400 could be part of another system. Forexample, the imaging system 400 could be implemented as part of anelectrophysiological experimentation system configured to apply optical,electrical, chemical, or other stimuli to a biological sample (e.g., asample of cultured or extracted neurons). The imaging system 400 couldprovide information about changes in the configuration of the biologicalsample in response to stimuli (e.g., by determining information aboutthe tissue related to the presence and/or location of probes in cells ofthe sample) and/or could provide information to inform to delivery ofstimuli. In some examples, the imaging system 400 could include multipleSLMs 410, light sources 420, light sensors 430, MDs 450, or otheradditional components. The imaging system 400 could include sensorsand/or be in communication with sensors configured to image otherproperties of a target environment (e.g., 405). Other configurations,operations, and applications of imaging systems as described herein areanticipated.

Other methods of configuring and/or operating a light source, lightsensor, SLM, MD, and/or other elements of an imaging system (e.g., toidentify and/or locate spatially and spectrally multiplexed probes in atarget) are anticipated.

IV. IDENTIFYING PROBES BY CORRELATION BETWEEN SPATIAL AND SPECTRALIMAGES

As described above, the colors and locations of fluorophores (e.g.,fluorophores of spatially and spectrally multiplexed probes) within anenvironment of interest can be determined by determining spectrographicinformation (e.g., a color) for a plurality of locations within theenvironment. Additionally or alternatively, light emitted from aparticular region (e.g., a small region, having a size corresponding toa diffraction limit of an optical system used to image the environment)of the environment of interest could be split and simultaneously used tospatially and spectrally image the particular region. The intensity oflight emitted from a particular fluorophore within the particular regioncan vary over time due to a variety of factors, e.g., due to blinking ofquantum dots or molecular fluorophores.

A pattern over time of such changes in intensity can be substantiallyindependent between different fluorophores within the environment. As aresult, time-varying waveforms of light detected from a location of aparticular fluorophore in spatial images of the particular region may becorrelated with time-varying waveforms of light detected fromwavelengths of light that correspond to the color of the particularfluorophore in spectral images of the particular region. Correlationscan be determined between such detected time-varying waveforms of lightreceived from different locations of a particular region of anenvironment and such detected time-varying waveforms of light receivedat different wavelengths from the particular region of the environmentand such correlations can be used to determined colors and/or locationsof fluorophores (e.g., fluorophores of probes) within the environment.

In order to generate such time-varying waveforms of light received fromdifferent locations and at different wavelengths from a particularregion of a target (e.g., a biological sample), light emitted from theparticular region of the environment can be split (e.g., by a partiallysilvered mirror, by a polarizing filter) into two portions and eachportion could be applied to a respective light sensors. The first lightsensor could receive a first portion of the emitted light such that thelight is imaged, in-focus, by an array of light-sensitive elements ofthe first light sensor. Each light-sensitive element of the first sensorcould be used to generate a time-varying waveform of light emitted froma respective location of the particular region of the target.

The second light sensor could receive a second portion of the emittedlight that has interacted with a chromatically dispersive element (e.g.,a prism, a diffraction grating, one or more Amici prisms, a spatiallight modulator (SLM) as described elsewhere herein). The light could bereceived by the second sensor, having been spectrally dispersed byinteraction with (e.g., reflection from, refraction through) thechromatically dispersive element, such that the light is detected by anarray of light-sensitive elements of the second light sensor. Such anarray could be a 2-dimensional array (e.g., such that the light could bedetected at different times having been spectrally dispersed indifferent directions and/or by different amounts by an SLM) or a1-dimensional array (e.g., the chromatically dispersive element could beconfigured to spectrally disperse the second portion of light in thedirection of the 1-dimensional array of elements). Each light-sensitiveelement of the second sensor could be used to generate a time-varyingwaveform of light emitted from the particular region of the target(e.g., from substantially the entire particular region) at a respectivewavelength.

FIG. 5A illustrates a particular region 500 of a target. Within theparticular region 500 of the target is a probe 505 that includes, inorder, a green fluorophore, Ga, a red fluorophore, Ra, and a bluefluorophore, Ba, disposed on a substantially linear backbone. Thus,spectrographic properties (e.g., colors) of locations of the particularregion 500 are such that red light is emitted from the location of Ra inresponse to illumination, green light is emitted from the location of Gain response to illumination, and blue light is emitted from the locationof Ba in response to illumination. The particular region 500 could beimaged by an imaging system as described elsewhere herein.

FIG. 5B illustrates a portion of a first image 510 a of the particularregion 500. This first image 510 a is taken of light received from thetarget that has been imaged by light-sensitive elements of a first lightsensor (e.g., a camera). The light-sensitive elements of the first lightsensor are arranged on a focal surface of the first light sensor and thereceived light is presented to the first light sensor (e.g., by opticsof an imaging system) such that the focal surface of the first lightsensor is conjugate to a focal surface passing through the particularregion 500. Thus, first image 510 a includes illuminated regions(indicated in FIG. 5B by three circles) illuminated by light emittedfrom the red, green, and blue fluorophores (Ra, Ga, and Ba) of the probe505. Light-sensitive elements of the first light sensor that correspondto the illuminated regions (illustrated in FIG. 5B as examplelight-sensitive elements 512 a, 514 a, 516 a) can be used to detect,over time, time-varying waveforms of light emitted from the red, green,and blue fluorophores (Ra, Ga, and Ba).

FIG. 5C illustrates example time-varying waveforms 522 a, 524 a, and 526a generated using the first 512 a, second 514 a, and third 516 a examplelight-sensitive elements, respectively. As the example light-sensitiveelements 512 a, 514 a, 516 a receive light from respective differentfluorophores, the time-varying waveforms 522 a, 524 a, 526 a may bedifferent and may correspond to the substantially independent changesover time of the amount of light emitted from each of the fluorophores(e.g., according to substantially random and independent blinking orother processes of the fluorophores). This is illustrated in FIG. 5C bythe example time-varying waveforms 522 a, 524 a, 526 a being different.

Note that time-varying waveforms could be generated for each of thelight-sensitive elements of a light sensor. In examples wherein each ofthe light-sensitive elements of the light sensor receives light from arespective location of a target (e.g., as illustrated in FIGS. 5A and5B), correlations between such time-varying waveforms could be used todetermine the location of fluorophores within the target. This couldinclude performing principal components analysis, independent componentsanalysis, clustering, determining pairwise correlations between thetime-varying waveforms and using the determined correlations todetermine sets of similar time-varying waveforms, or some other methodor methods to determine similarities between sets of time-varyingwaveforms and then using such determined similarities to determine thelocation of fluorophores within a target. Additionally or alternatively,pattern matching or other techniques could be used to determine thecenters of illuminated regions of one or more images (e.g., 510 a) ofthe target. Note that illuminated regions of a light sensor thatcorrespond to respective different fluorophores of a probe in a targetmay significantly overlap. In such examples, the time-varying waveformof light generated using a particular light-sensitive element of a lightsensor may include a combination of waveforms corresponding to thechange over time of the light emitted from multiple differentfluorophores.

FIG. 5D illustrates a portion of a second image 510 b of the particularregion 500. This second image 510 b is taken of light received from theparticular region 500 that has been spectrally dispersed by achromatically dispersive element. With respect to the example secondimage 510 b, the chromatically dispersive element is configured tospectrally disperse light received from the particular region 500 in ahorizontal direction such that longer wavelengths of light are receivedby light-sensitive elements of the light sensor to the left in FIG. 5Dand such that shorter wavelengths of light are received bylight-sensitive elements of the light sensor to the right in FIG. 5D.Thus, the second image 510 b includes illuminated regions (indicated inFIG. 5D by three vertical bars, whose wavelengths are indicated by R, G,and B along the horizontal axis of the image 510 b) due to illuminationof corresponding regions of a light sensor by spectrally dispersed lightfrom the red, green, and blue fluorophores (Ra, Ga, and Ba),respectively, of the probe 505. Light-sensitive elements of the secondlight sensor that correspond to the illuminated regions (illustrated inFIG. 5D as example light-sensitive elements 512 b, 514 b, 516 b) can beused to detect, over time, time-varying waveforms of light emitted fromthe red, green, and blue fluorophores (Ra, Ga, and Ba).

FIG. 5E illustrates example time-varying waveforms 522 b, 524 b, and 526b generated using the first 512 b, second 514 b, and third 516 b examplelight-sensitive elements, respectively. As the example light-sensitiveelements 512 b, 514 b, 516 b receive light from respective differentfluorophores, the time-varying waveforms 522 b, 524 b, 526 b may bedifferent and may correspond to the substantially independent changesover time of the amount of light emitted from each of the fluorophores(e.g., according to substantially random and independent blinking orother processes of the fluorophores). This is illustrated in FIG. 5C bythe example time-varying waveforms 522 b, 524 b, 526 b being different.Note that, as the time-varying waveforms 522 b, 524 b, 526 b correspondto the changes over time in the amount of light emitted from thefluorophores of the probe 505, they may be similar to time-varyingwaveforms of light received, by light-sensitive elements of the firstlight sensor, from corresponding fluorophores of the probe 505.

This is illustrated, by way of example, in FIGS. 5C and 5E bytime-varying waveforms 522 a and 524 b being similar. Time-varyingwaveform 522 a corresponds to changes over time in the amount of lightreceived, by light-sensitive element 512 a of the first light sensor,from the green fluorophore, Ga, of the probe 505. The green fluorophoreemits green light; thus, time-varying waveform 524 b corresponds tochanges over time in the amount of light received, by light-sensitiveelement 524 b of the second light sensor, from the green fluorophore. Asa result, time-varying waveforms 522 a and 524 b are similar and mayhave a high degree of correlation or some other metric of similarity.Similarly, time-varying waveforms 524 a and 522 b are similar (due tocorrespondence with changes over time in the amount of light emittedfrom the red fluorophore, Ra) and time-varying waveforms 526 a and 526 bare similar (due to correspondence with changes over time in the amountof light emitted from the blue fluorophore, Ba).

Correlations between time-varying waveforms generated usinglight-sensitive elements of the first light sensor and second lightsensor could be determined and used to determine the color and/orlocation of fluorophores within the target. This could includeperforming principal components analysis, independent componentsanalysis, clustering, determining pairwise correlations between thetime-varying waveforms and using the determined correlations todetermine sets of similar time-varying waveforms, or some other methodor methods to determine similarities between sets of time-varyingwaveforms and then using such determined similarities to determinecolors of fluorophores in the target. This could include determiningthat a determined correlation between a particular time-varying waveformof light generated using a light-sensitive element of the first lightsensor (e.g., a time-varying waveform of light corresponding to lightemitted from a particular location of a target that includes afluorophore) and a particular time-varying waveform of light generatedusing a light-sensitive element of the second light sensor (e.g., atime-varying waveform of light corresponding to a wavelength of lightemitted from the fluorophore in the target) is greater than a threshold.If the particular time-varying waveform generated by the first lightsensor corresponds to a determined location of a fluorophore, thecorrelation being greater than the threshold could be used to determinethat the color of the fluorophore includes a wavelength of lightcorresponding to the particular time-varying waveform generated by thesecond light sensor.

Note that different fluorophores location in a particular region of atarget that is being imaged (e.g., as shown in FIG. 5A) may emit lightat the same wavelength. This could be due to the different fluorophoreshaving the same color and/or being the same type of fluorophore.Additionally or alternatively, one or more of the fluorophores couldemit light at multiple different wavelengths (e.g., a first quantum dotcould emit light at a red wavelength and a blue wavelength, while asecond quantum dot could emit light at a red wavelength and a greenwavelength). In such examples, the time-varying waveform of lightgenerated using a particular light-sensitive element of a light sensor(e.g., the second light sensor) may include a combination of waveformscorresponding to the change over time of the light emitted from multipledifferent fluorophores.

Further, note that, while the second light sensor discussed incombination with FIGS. 5A-E includes a two-dimensional array oflight-sensitive elements, the second light sensor could alternatively beconfigured to have a one-dimensional array of light-sensitive elementsconfigured to generate respective time-varying waveforms of lightreceived from a particular region of a target by the light-sensitiveelements at respective different wavelengths. In examples wherein thesecond light sensor comprises a two-dimensional array of light-sensitiveelements, a chromatically dispersive element that is used to spectrallydisperse the light received by the second light sensor could beoperated, during different period of time, to disperse the light indifferent directions, by different amounts, or according to some otherconsideration. Two-dimensional images generated by the second lightsensor in such an example could be used (e.g., via a process ofdeconvolution) to provide spectrographic information for locations ofthe particular region of the target.

Time varying waveforms could be generated, using each light-sensitiveelement of a light sensor, in a variety of ways. In some examples, eachlight-sensitive element could be connected to a respective amplifier,digitizer, filter, or other component configured to generate atime-varying waveform using the light-sensitive element. Alternatively,light-sensitive elements of a light sensor could be operated torepeatedly image a target. For example, the light sensor could include acharge-coupled device (CCD), and imaging the target using the lightsensor could include operating the CCD to sequentially transfer chargesaccumulated by the light-sensitive elements (e.g., pixels) of the CCD toone or more digitizers to be sampled. Generating a time-varying waveformfor a particular light-sensitive element of the light sensor could, insuch examples, include using a plurality of such images to determine theintensity of light received by the particular light-sensitive elementover time, at a plurality of points in time corresponding to the timingof generation of the plurality of images. Further, the first and secondlight sensors could be part of the same device, e.g., the same CCD. Thatis, light-sensitive elements of the first and second light sensor couldcomprise respective first and second sets of light-sensitive elements ofthe same light-sensing device. In such an example, optical elements ofan imaging system used to image a target using the first and secondlight sensors could be configured to split the light received from atarget into first and second portions, spectrally disperse the secondportion, and present the first and second portions to respective set oflight-sensitive elements (e.g., respective different areas) of a singlelight-sensing device.

In order to produce time-varying waveforms of light received fromdifferent locations of a particular region of a target and to producetime-varying waveforms of light received from the particular region ofthe target at different wavelengths, as described above, that can beused to identify probes in the target by determining the locations andcolors of fluorophores of the probes in the target, a variety ofdifferent optical elements could be incorporated into an imaging system.Such an imaging system could include various sources of illumination,optical systems, apertures, light sensors, or other elements configuredto illuminate a particular region of a target, receive lightresponsively emitted from the particular region of the target (e.g.,from fluorophores in the particular region), split the received lightinto first and second portions, apply a first portion to be spatiallyimaged by a first light sensor, and apply a second portion to bespectrally dispersed by a chromatically dispersive element (e.g., aprism, an SLM) and spectrally imaged by a second light sensor. Such animaging system could include apertures, optics, micromirror devices,Nipkow discs, or other elements configured to selectively receive lightfrom the particular region of the target and/or to selectivelyilluminate the particular region of the target. This could include usingtechniques and/or apparatus for confocally imaging the particular regionof the target.

FIG. 6 illustrates in cross-section elements of an example imagingsystem 600 configured to image a target 605. The system 600 includes alight source 620 (e.g., a laser), a first light sensor 630 (illustratedas a plane of light-sensitive elements located on a focal plane 637 ofthe first light sensor 630), a second light sensor 670 (illustrated as aplane of light-sensitive elements), a micromirror device (MD) 650, achromatically dispersive element 610 (e.g., a prism, one or more Amiciprisms, an SLM, a diffraction grating), and an optical system (includingan objective 641, first 675, second 676, and third 644 relay lenses, adichroic mirror 645, a beam splitter 643, and an optical sink 625)configured to direct light to and from the target 605 and between theelements of the system 600. The system 600 additionally includes a stage660 to which the target 605 is mounted. Note that the MD 650 and firstlight sensor 630 comprise two-dimensional arrays of micromirrors andlight-sensitive elements, respectively. The second light sensor 670 maycomprise a two-dimensional or one-dimensional array of light-sensitiveelements. Further, note that the optical system (e.g., 641, 643, 644,645, 675, 676) and chromatically dispersive element 610 are configuredto direct light between the target 605, MD 650, and first light sensor630 such that locations on the focal surfaces 657, 637 of the MD 650 andfirst light sensor 630 correspond to respective locations on the focalsurface 607 passing through a particular region 609 in the target 605.

The system 600 illuminates a particular region 609 in the target 605 byemitting a first illumination 621 from the light source 620 andreflecting the first illumination 621 from the dichroic mirror 645toward the MD 650. A selected mirror 651 of the MD 650 that has alocation on a focal surface 657 of the MD 650 corresponding to theparticular region 609 is controlled to have a first angle to reflect thefirst illumination 621 toward the target 605 as confocal illumination622 via the objective 641. Other mirrors 653 of the MD 650 arecontrolled to have a second angle to reflect the remainder of the firstillumination 621 as waste illumination 623 toward the optical sink 625to be absorbed. As illustrated, a single mirror (651) is controlled toilluminate (and to receive light from) a corresponding region 609 of thetarget 605; however, additional mirrors (e.g., selected from othermirrors 653) could be operated simultaneously, sequentially, oraccording to some other scheme to illuminate (and to receive light from)corresponding additional regions of the target 605.

The system 600 receives light (including conjugate light 672) emittedfrom the target 605 (e.g., from the particular region 609) in responseto illumination via the objective 641. The conjugate light 672 is split,by the beam splitter 643, into first and second portions ofillumination. The first portion of the conjugate light is presented,in-focus, to a specified region 631 on a focal surface 637 of the firstlight sensor 630 corresponding to the particular region 609 (e.g., to aregion of the first light sensor having one or more light-sensitiveelements and/or pixels of the first light sensor 630). The secondportion of the conjugate light is applied to the chromaticallydispersive element 610 to spectrally disperse the second portion oflight. The spectrally dispersed portion of light is then presented tothe second light sensor 670. Such manipulations of the conjugate light672 (e.g., reflections, splitting, spectral dispersions) are provided byrelay optics 675, 676, 644, beam splitter 643, chromatic dispersiveelement 610, or by some other optical element(s).

Note that, while the chromatically dispersive element 610 is illustratedin FIG. 6 as being a transmissive element (that is, an element thatspectrally disperses light that is transmitted through the chromaticallydispersive element 610), an imaging system as described herein couldinclude a reflective, a refractive, a diffractive, or some other varietyof chromatically dispersive element and/or a combination ofchromatically dispersive elements. In some examples, the chromaticallydispersive element 610 could include an SLM as described elsewhereherein that can be operated to control a direction, a magnitude, or someother properties of the spectral dispersion applied to light that ispresented to the second light sensor 670.

The location of the particular region 609 within the target 605 could becontrolled (e.g., to allow imaging of elements of the target 605 atdifferent depths within the target 605). In some examples, the stage 660could be actuated relative to other elements of the system 600 such thata location of the target 605 in one or more dimensions could becontrolled. For example, the stage 660 could be actuated in a directionparallel to the direction of the illumination (i.e., in the verticaldirection of FIG. 6) such that the location (e.g., the depth) of theparticular region 609 within the target 605 could be controlled.Actuation of the stage 660 could include one or more piezo elements,servos motors, linear actuators, galvanometers, or other actuatorsconfigured to control the location of the stage 660 (and a target 605mounted on the stage 660) relative to element(s) (e.g., 641) of thesystem 600.

Note that the configuration and/or operation of the system 600 toilluminate and to receive conjugate light from a particular region 609of a target 605 is intended as a non-limiting example. Alternatively, alarger and/or differently-shaped region of the target could beilluminated by operating the mirrors 651, 653 of the MD 650 according toa different set of controlled angles than those illustrated. Forexample, a plurality of spatially separated regions proximate of thetarget 605 could be illuminated and imaged simultaneously by controllinga corresponding plurality of spatially separated mirrors of the MD 650to reflect the first illumination 621 toward the plurality of regions ofthe target 605. A separation distance between such spatially separatedregions could be greater than a specified distance such that light fromeach region does not interfere with detection of spectral and/or spatialinformation for other regions using the first 630 and/or second 670light sensors, respectively (e.g., such that spots of light projectedonto the second light sensor 670 from each of the spatially separatedregions following being spectrally dispersed by the chromaticallydispersive element 610 do not overlap).

Further, the size of the illuminated regions could be sufficiently large(e.g., could be illuminated by sufficiently many mirrors of the MD 650)that sufficiently many pixels of the first 630 and/or second 670 camerasare illuminated to allow statistical analysis to be performed (e.g., toallow sufficiently many correlations to be determined to determinecolors and locations of fluorophores in the target 605). The size of theilluminated regions could also be sufficiently small that spectralinformation for different fluorophores is smeared, across pixels of thesecond light sensor 670, by a sufficiently small amount by spatialdistances between different fluorophores in each illuminated region thatspectral information (e.g., colors) can be determined for all of thefluorophores in each illuminated region.

The mirrors 651, 653 of the MD 650 could be controlled according to someother pattern, e.g., to approximate some other coded aperture on thefocal surface 657 of the MD 650. Further, the light source 620 couldemit illumination at a controllable wavelength (e.g., illumination thatis substantially monochromatic, but having a wavelength that can bealtered by operation of the light source) and spectrographic informationcould be determined for regions of the target 605 based on images of thetarget 605 generated when the target 605 is illuminated by differentwavelengths of light (e.g., to generate a corresponding plurality ofemission spectra for fluorophores of probes in the target correspondingto the different wavelengths of illumination).

The system 600 could be operated in a variety of ways to provideconfocal, hyperspectral, or other types of images of the target 605. Forexample, the system could be operated during a number of periods of timeto illuminate respective particular regions of the target (e.g., bycontrolling respective specified sets of mirrors of the DM to have firstor second angles), to image light received from the target 605 using thesecond 670 and first 630 light sensors, respectively, or to operate someother element(s) of the system 600 over time to identify and/or locateprobes in a target or according to some other application.

Note that non-conjugate light received from a target (e.g., light fromthe target 605 that is reflected by the second set of mirrors 653 of theMD 650) could be used to determine spatial or spectral information aboutfluorophores and/or probes in the target (e.g., to generate time-varyingwaveforms of light received from different locations of a particularregion of the target and/or to generate time-varying waveforms of lightreceived from the particular region of the target at differentwavelengths). For example, an MD could be used to partition lightreceived from a particular region of a target such that a first portionis presented, in-focus, to a first light sensor (e.g., to spatiallyimage the particular region) and such that a second portion isspectrally dispersed and presented to a second light sensor (e.g., tospectrally image the particular region).

Such a scenario is illustrated by way of example in FIG. 7. FIG. 7illustrates in cross-section elements of an example imaging system 700configured to image a target 705 in order to, e.g., identify and/orlocate spatially and spectrally multiplexed probes in the target 705.The system 700 includes a light source 720 (e.g., a laser), a secondlight sensor 730 (illustrated as a plane of light-sensitive elementslocated on a focal plane 737 of the second light sensor 730), a firstlight sensor 770 (illustrated as a plane of light-sensitive elementslocated on a focal plane 777 of the first light sensor 770), amicromirror device (MD) 750, a spatial light modulator (SLM) 710, and anoptical system (including an objective 741, first 743, second 744, third775, and fourth 776 relay lenses, a dichroic mirror 745, and an opticalsink 725) configured to direct light to and from the target 705 andbetween the elements of the system 700. The system 700 additionallyincludes a stage 760 to which the target 705 is mounted. Note that theMD 750 and second 730 and first 770 light sensor comprisetwo-dimensional arrays of micromirrors and light-sensitive elements,respectively. Further, note that the optical system (e.g., 741, 743,744, 745, 775, 776) and SLM 710 are configured to direct light betweenthe target 705, MD 550, and second 730 and first 770 light sensors suchthat locations on the focal surfaces 757, 737, 777 of the MD 750 andlight sensors 730, 770 correspond to respective locations on a focalsurface 707 that passes through a particular region 709 in the target705.

The system 700 illuminates a particular region 709 in the target 705 byemitting a first illumination 721 from the light source 720 andreflecting the first illumination 721 from the dichroic mirror 745toward the MD 750. A selected mirror 751 of the MD 750 that has alocation on a focal surface 757 of the MD 750 corresponding to thespecified region 709 is controlled to have a first angle to reflect thefirst illumination 721 toward the target 705 as confocal illumination722 via the objective 741. Other mirrors 753 of the MD 750 arecontrolled to have a second angle to reflect the remainder of the firstillumination 721 as waste illumination 723 toward the optical sink 725to be absorbed. As illustrated, a single mirror (751) is controlled toilluminate (and to receive light from) a corresponding particular region709 of the target 705; however, additional mirrors (e.g., selected fromother mirrors 753) could be operated simultaneously, sequentially, oraccording to some other scheme to illuminate (and to receive light from)corresponding additional regions of the target 705.

The system 700 receives light (including conjugate light 772) emittedfrom the target 705 (e.g., from the particular region 709) in responseto illumination via the objective 741. The conjugate light 772 isdirected, in-focus, to a specified region 771 on a focal surface 777 ofthe first light sensor 770 corresponding to the particular region 709(e.g., to a region of the first light sensor having one or morelight-sensitive elements and/or pixels of the first light sensor 770).Such light is directed to the first light sensor 770 from the MD 750 viarelay optics 775, 776 or via some other optical element(s).

The system 700 also receives non-conjugate light 732 emitted from thetarget via the objective 741. The non-conjugate light 732 arrives at theMD 750 and is reflected by mirrors of the MD 750 that are controlled tohave the second angle (e.g., 753) toward the SLM 710. The first relaylens 743 (and/or some other optical elements of the system 700)collimates the received light and presents the substantially collimatedlight to the SLM 710. The SLM 700 reflects the non-conjugate light 732as spectrally dispersed light 733 toward the second relay lens 744 thatis configured to present the spectrally dispersed light 733 to a focalsurface 737 of the second light sensor 730. The SLM 710 is configuredand/or operated such that the spectrally dispersed light 733 isspectrally dispersed relative to the non-conjugate light 732 in acontrolled manner such that spectrographic information of one or moreparticular regions of the target 705 and/or of the non-conjugate light732 can be detected or determined (e.g., based on a plurality oftime-varying waveforms of light received from the particular region 709at respective different wavelengths by respective differentlight-sensitive elements of the second light sensor 730). In someexamples, the spectrally dispersed light 733 is spectrally dispersed ina manner related to an electronically controlled direction, magnitude,and/or some other property of a spatial gradient in the refractive indexof a layer of the SLM 710.

Note that the configuration and/or operation of the system 700 toilluminate and to receive conjugate light from a particular region 709of the target 705 is intended as a non-limiting example. Alternatively,a larger and/or differently-shaped region of the target could beilluminated by operating the mirrors 751, 753 of the MD 750 according toa different set of controlled angles than those illustrated. Forexample, a plurality of spatially separated regions of the target 705could be illuminated and imaged simultaneously by controlling acorresponding plurality of spatially separated mirrors of the MD 750 toreflect the first illumination 721 toward the plurality of the regionsof the target 705. The mirrors 751, 753 of the MD 750 could becontrolled according to some other pattern, e.g., to approximate someother coded aperture on the focal surface 757 of the MD 750. Further,the light source 720 could emit illumination at a controllablewavelength (e.g., illumination that is substantially monochromatic, buthaving a wavelength that can be altered by operation of the lightsource) and spectrographic information could be determined for regionsof the target 705 based on images of the target 705 and/or time-varyingwaveforms of light received from the target 705 generated when thetarget 705 is illuminated by different wavelengths of light (e.g., togenerate a corresponding plurality of emission spectra for one or morefluorophores of a probe corresponding to the different wavelengths ofillumination).

The location of the particular region 709 within the target 705 could becontrolled (e.g., to allow imaging of elements of the target 705 atdifferent depths within the target 705). In some examples, the stage 760could be actuated relative to other elements of the system 700 such thata location of the target 705 in one or more dimensions could becontrolled. For example, the stage 760 could be actuated in a directionparallel to the direction of the illumination (i.e., in the verticaldirection of FIG. 7) such that the location (e.g., the depth) of theparticular region 709 within the target 705 could be controlled.Actuation of the stage 760 could include one or more piezo elements,servos motors, linear actuators, galvanometers, or other actuatorsconfigured to control the location of the stage 760 (and a target 705mounted on the stage 760) relative to element(s) (e.g., 741) of thesystem 700.

The system 700 could be operated in a variety of ways to provideconfocal, hyperspectral, or other types of images of the target 705. Forexample, the system could be operated during a number of specifiedperiods of time to illuminate different regions of the target (e.g., bycontrolling respective specified sets of mirrors of the DM to have firstor second angles, by controlling an actuated stage 760), toelectronically control a gradient of refractive index across arefractive layer of the SLM to have respective different specifiedmagnitude(s) or direction(s) or to control the refractive index ofelement(s) of the SLM according to some other patterns, to imageconjugate or non-conjugate light received from the target 705 using thefirst 770 and second 730 light sensors, respectively, or to operate someother element(s) of the system 700 over time according to anapplication.

Other methods of configuring and/or operating a light source, lightsensor(s), one or more apertures, SLM, MD, and/or other elements of animaging system (e.g., to identify and/or locate spatially and spectrallymultiplexed probes in a target) are anticipated

V. EXAMPLE ELECTRONICS OF AN IMAGING APPARATUS

FIG. 8 is a simplified block diagram illustrating the components of animaging system 800, according to an example embodiment. Imaging system800 and/or elements thereof may take the form of or be similar to one ofthe example systems or elements 300, 400, 600, 700 shown in FIGS. 3, 4,6, and 7 or of some other systems. For example, imaging system 800and/or elements thereof may take the form of or be similar to one of theexample systems or elements shown in FIGS. 13, 14, 15, 16, 17, 18, 19,20, 21, 25, 28, 29, 30, 31, 35, 36, 38, and 39. Imaging system 800 maytake a variety of forms, such as a wall, table, ceiling, orfloor-mounted device. Imaging system 800 may take the form of abench-top or table-top device (e.g., a bench-top microscope). Imagingsystem 800 and/or elements thereof could also take the form of a system,device, or combination of devices that is configured to be part ofanother device, apparatus, or system. For example, imaging system 800 orelement(s) thereof (e.g., spatial light modulator 803) could take theform of a system or element configured to be mounted to or otherwisedisposed as part of some other imaging system (e.g., imaging system 800and/or the spatial light modulator 803 or other elements thereof couldbe configured to be part of a confocal microscope or other imagingsystem, e.g., to spectrally disperse one or more beams or fields oflight of the imaging system in an electronically-controllable manner).Imaging system 800 could take the form of a system configured tocontents of an industrial environment, medical environment, scientificenvironment, or some other environment. Imaging system 800 also couldtake other forms.

In particular, FIG. 8 shows an example of an imaging system 800 having alight source 801, a first light sensor 802, a spatial light modulator(SLM) 803, a micromirror device (MD) 806, a second light sensor 807, anoptical system 805, a stage actuator 808, a user interface 820,communication system(s) 830 for transmitting data to a remote system,and controller 810. The components of the imaging system 800 may bedisposed on or within a mount or housing or on some other structure formounting the system to enable stable imaging or other functions relativeto a target of interest, for example, a biological sample mounted to astage (e.g., a stage having a location relative to other elements of theimaging system 800 that is actuated in at least one dimension by thestage actuator 808). The imaging system 800 could include additionalcomponents, for example, a perfusion pump configured to provide aeratedor otherwise chemically specified perfusate to a cell culture or otherbiological sample comprising a target of the imaging system 800, one ormore electrophysiological or optogenetic stimulators and/or sensors, anintegrated circuit test rig, or some other instrument(s) or othercomponent(s) according to an application.

The light source 801, light sensors 802, 807, optical system 805, SLM803, MD 806, and/or stage actuator 808 could be configured and/ordisposed as part of the imaging device 800 as described elsewhere hereinfor similar elements. The optical system 805 is configured to directlight emitted by the light source 801 to illuminate one or more regionsof a target (e.g., via reflection from one or more mirrors of the MD806). The optical system 805 is further configured to receive lightresponsively emitted from the target and to direct such light and/orcomponents of such light (e.g., a conjugate component of the receivedlight, a non-conjugate component of the received light, an otherwisepartitioned portion of the received light) to one or both of the lightsensors 801, 807 (e.g., via reflection from one or more mirrors of theMD 806, via reflection from, transmission through, or some otherchromatically disperse interaction with the SLM 803 and/or some otherchromatically dispersive element(s) of the imaging system 800). Theoptical system 805 is configured to direct such light between elements(e.g., 802, 806, 807) of the imaging system 800 such that focal surfacesof one or more such elements (e.g., a focal surface of the first lightsensor 801 on which is disposed light-sensitive elements of the lightsensor, a focal surface of the MD 806 on which is disposed mirrors ofthe MD 806) are optically conjugate with each other and/or with a focalsurface on or within a target of the imaging system 800.

Controller 810 may be provided as a computing device that includes oneor more processors 811. The one or more processors 811 can be configuredto execute computer-readable program instructions 814 that are stored ina computer readable data storage 812 and that are executable to providethe functionality of an imaging system 800 as described herein.

The computer readable data storage 812 may include or take the form ofone or more non-transitory, computer-readable storage media that can beread or accessed by at least one processor 811. The one or morecomputer-readable storage media can include volatile and/or non-volatilestorage components, such as optical, magnetic, organic or other memoryor disc storage, which can be integrated in whole or in part with atleast one of the one or more processors 811. In some embodiments, thecomputer readable data storage 812 can be implemented using a singlephysical device (e.g., one optical, magnetic, organic or other memory ordisc storage unit), while in other embodiments, the computer readabledata storage 812 can be implemented using two or more physical devices.

The program instructions 814 stored on the computer readable datastorage 812 may include instructions to perform any of the methodsdescribed herein. For instance, in the illustrated embodiment, programinstructions 814 include an illumination and acquisition module 815 anda probe detection module 816.

The illumination and acquisition module 815 can include instructions foroperating the light source 801, first light sensor 802, SLM 803, MD 806,second light sensor 807, and/or stage actuator 808 to enable any of thefunctions or applications of an imaging system to identify, determine anorientation of, and/or locate spatially and spectrally multiplexedprobes in a target and/or to hyperspectrally image, confocally image, orotherwise image or optically interact with a target as described herein.Generally, instructions in the illumination and acquisition module 815provide methods of operating the light source 801 and/or MD 806 toilluminate one or more regions of a target with light at one or morespecified wavelengths during one or more respective periods of time.Instructions in the illumination and acquisition module 815 furtherprovide methods of operating the SLM 803 and/or some other chromaticallydispersive element(s) to spectrally disperse light directed toward theSLM 803 according to one or more specified directions, magnitudes, orother properties of dispersion of light during one or more respectiveperiods of time (e.g., periods of time synchronous with and/oroverlapping periods of time of operation of the MD 806 and/or lightsource 801).

Instructions in the illumination and acquisition module 815 furtherdescribe methods of operating the light sensor(s) 801, 807 to generateimages, time-varying waveforms, or other information about lightreceived from illuminated regions of a target via the optical system805, micromirror device 806, and/or SLM 803 during one or more periodsof time (e.g., periods of time of operation of the MD 806, SLM 803,light source 801, or other components of the imaging system 800). Insome examples, generating an image and/or one or more time-varyingwaveforms of received light using the light sensor(s) 801, 807 couldinclude reading out information (e.g., values or signals describing ofrelated to the intensity or other property of light detected bylight-sensitive elements of the light sensor(s) 801, 807. In suchexamples, a particular light-sensitive element or set of light-sensitiveelements of the light sensor could be substantially unable to detectlight when being read out. For example, one or both of the lightsensor(s) could be CMOS cameras configured to have a global shutter(i.e., to read out an entire frame of image data from the light sensorat a time) and/or to have a rolling shutter (i.e., to read out a row ofimage data from the light sensor at a time). In such embodiments, theillumination and acquisition module 815 could describe operations of anMD 806 or other elements to not illuminate regions of a targetcorresponding to locations (e.g., light-sensitive elements) of the lightsensor(s) that are not able to detect light from such regions (e.g.,light-sensitive elements that are being read out). Other operations,functions, and applications of the light source 801, first light sensor802, SLM 803, MD 806, second light sensor 807, stage actuator 808,and/or of other components of the imaging system 800 as described hereincould be implemented as program instructions in the illumination anddetection module 815.

The probe detection module 816 can include instructions for identifying,locating, determining the orientation of, or determining some otherinformation about probes disposed within a target. This could includedetermining colors and locations of fluorophores of such probes andusing such determined colors and locations to identify and/or locateprobes within the target, e.g., by matching a pattern of fluorophorelocations and/or an order of fluorophore colors to a template pattern ororder that corresponds to a particular type of probe. Suchdeterminations could be based on one or more images, time-varyingwaveforms of light, or other information of signals generated by thelight sensor(s) 801, 807.

For example, the probe detection module 816 can include instructions fordetermining information about colors and locations of fluorophores in atarget based on one or more images of spectrally dispersed lightreceived from the target. Such a determination could include processesas described herein (e.g., a process of deconvolution, a process similarto the process described by example in relation to FIGS. 2A-D, someother process(es)). Such processes could be based on a description ofcorrespondences between the location of light-sensitive elements of thelight sensor(s) 801, 807 and corresponding locations on or within thetarget. Such correspondences could be wavelength-dependent and could bedetermined based on a model of the imaging system 800 (e.g., based onthe magnitude and direction of a gradient of refractive index of arefractive layer across the SLM 803 during one or more periods of timecorresponding to images generated by the light sensor(s) 801, 807)and/or on an empirical measurement of the properties of the system 800(e.g., based on a set of images of a calibration target having knownspectrographic information/content or some other calibration informationor procedure).

In another example, the probe detection module 816 can includeinstructions for determining information about colors and locations offluorophores in a target based on one or more pluralities oftime-varying waveforms of light received form the target and generatedusing the light sensors 801, 807. In a particular example, the opticalsystem 805 could be configured to receive light emitted from aparticular region of a target (e.g., from a region that is beingilluminated by via an aperture of the optical system 805 and/or via asynthetic aperture formed from one or more mirrors of the MD 806) and topresent portion of such light to the first 801 and second 807 lightsensors. The optical system 805 could present a first portion of thereceived light to the first light sensor 801 such that eachlight-sensitive element of the first light sensor 801 receives lightfrom a respective location of the particular region (e.g., such that thelight-sensitive elements are location on a focal surface of the firstlight sensor 801 that is conjugate to a focal surface passing throughthe particular region of the target). The optical system could present asecond portion of the received light to the second light sensor 807, viaa chromatically dispersive element (e.g., via reflection from the SLM803) such that each light-sensitive element of the second light sensor807 receives light from the particular region at a respectivewavelength.

Time varying waveforms of light received by light-sensitive elements ofthe light sensors 801, 807 could be generated and used by the probedetection module 816 to determine colors and locations of fluorophoresof probes in the target. Such a determination could include processes asdescribed herein (e.g., a process of determining pairwise correlationsbetween time-varying waveforms of light, by performing a principalcomponents analysis, an independent components analysis, by performing aclustering analyses, or by performing some other process(es)).

Some of the program instructions of the illumination and acquisitionmodule 815 and/or probe detection module 816 may, in some examples, bestored in a computer-readable medium and executed by a processor locatedexternal to the imaging system 800. For example, the imaging system 800could be configured to illuminate and to receive light from a target(e.g., a biological sample) and then transmit related data to a remoteserver, which may include a mobile device, a personal computer, thecloud, or any other remote system, for further processing (e.g., for thedetermination of spectrographic information of one or more regions ofthe target, for identifying the region of the target and/or contentsthereof based on the determined spectrographic content, to identifyand/or determine locations of probes in the target).

User interface 820 could include indicators, displays, buttons,touchscreens, head-mounted displays, and/or other elements configured topresent information about the imaging system 800 to a user and/or toallow the user to operate the imaging system 800. Additionally oralternatively, the imaging system 800 could be configured to communicatewith another system (e.g., a cellphone, a tablet, a computer, a remoteserver) and to present elements of a user interface using the remotesystem. The user interface 820 could be disposed proximate to the lightsource 801, first light sensor 802, SLM 803, MD 806, second light sensor807, stage actuator 808, controller 810, or other elements of theimaging system 800 or could be disposed away from other elements of theimaging system 800 and could further be in wired or wirelesscommunication with the other elements of the imaging system 800. Theuser interface 820 could be configured to allow a user to specify someoperation, function, or property of operation of the imaging system 800.The user interface 820 could be configured to present an image of atarget (e.g., an image of the location or distribution of one or moretypes of probes within the target) generated by the imaging system 800or to present some other information to a user. Other configurations andmethods of operation of a user interface 820 are anticipated.

Communication system(s) 830 may also be operated by instructions withinthe program instructions 814, such as instructions for sending and/orreceiving information via a wireless antenna, which may be disposed onor in the imaging system 800. The communication system(s) 830 canoptionally include one or more oscillators, mixers, frequency injectors,etc. to modulate and/or demodulate information on a carrier frequency tobe transmitted and/or received by the antenna. In some examples, theimaging system 800 is configured to indicate an output from thecontroller 810 (e.g., one or more images of a target) by transmitting anelectromagnetic or other wireless signal according to one or morewireless communications standards (e.g., Bluetooth, WiFi, IrDA, ZigBee,WiMAX, LTE). In some examples, the communication system(s) 830 couldinclude one or more wired communications interfaces and the imagingsystem 800 could be configured to indicate an output from the controller810 by operating the one or more wired communications interfacesaccording to one or more wired communications standards (e.g., USB,FireWire, Ethernet, RS-232).

The computer readable data storage 812 may further contain other data orinformation, such as contain calibration data corresponding to aconfiguration of the imaging system 800, a calibration target, or someother information. Calibration, imaging, and/or other data may also begenerated by a remote server and transmitted to the imaging system 800via communication system(s) 830.

VI. EXAMPLE METHODS

FIG. 9 is a flowchart of an example method 900 for operating elements ofan imaging system to identify spatially and spectrally multiplexedprobes in a target environment and/or to provide some other functionsand/or applications of the imaging system. The method 900 includesgenerating, using a plurality of light-sensitive elements of a firstlight sensor, a first plurality of respective time-varying waveforms oflight that is emitted from respective different locations of aparticular region of a target (902). This could include generating aplurality of images of the target, during a respective plurality ofperiods of time, and determining each of the time-varying waveformsbased on the intensity of light received by a respective light-sensitiveelement of the first light sensor in each of the generated images.Alternatively, generating a time-varying waveform of light received by aparticular light-sensitive element of the first light sensor couldinclude operating an amplifier, one or more filters, a digitizer, orsome other elements to generate a time-varying waveform using theparticular light-sensitive element. The light emitted from theparticular region of the target could be provided to the first lightsensor via an optical system, e.g., such that a focal surface of thefirst light sensor is conjugate to a focal surface passing through theparticular region of the target. This could include the emitted lightpassing through an aperture of the optical system (e.g., an aperture ofa Nipkow disk) and/or being reflected by a virtual aperture formed froma set of actuated mirrors of a micromirror device.

The method 900 additionally includes generating, using a plurality oflight-sensitive elements of a second light sensor, a second plurality ofrespective time-varying waveforms of light at respective differentwavelengths that is emitted from the particular region of a target(904). This could include generating a plurality of one- ortwo-dimensional images of the target, during a respective plurality ofperiods of time, and determining each of the time-varying waveformsbased on the intensity of light received by a respective light-sensitiveelement of the second light sensor in each of the generated images.Alternatively, generating a time-varying waveform of light received by aparticular light-sensitive element of the second light sensor couldinclude operating an amplifier, one or more filters, a digitizer, orsome other elements to generate a time-varying waveform using theparticular light-sensitive element. The light emitted from theparticular region of the target could be provided to the second lightsensor via an optical system that includes one or more chromaticallydispersive elements (e.g., a prism, an SLM). Such an optical systemcould be configured such light of different wavelengths that is emittedfrom the particular region is spread, according to the wavelength of thelight, across a number of the light-sensitive elements of the secondlight sensor.

The method 900 additionally includes determining correlations betweentime-varying waveforms of the first plurality of time-varying waveformsand time-varying waveforms of the second plurality of time-varyingwaveforms (906). This could include determining, for each combination oftime-varying waveforms for the first plurality of time-varying waveformsand the second plurality of time-varying waveforms, or for a subset ofsuch combinations, a correlation coefficient or some other metric ofsimilarity (e.g., an inner product). This (906) could include performinga principal components analysis, an independent components analysis, aclustering analysis, or some other method to determine sets oftime-varying waveforms within or between the first and secondpluralities of time-varying waveforms that are similar.

The method 900 additionally includes determining, based on thedetermined correlations, locations and colors of two or morefluorophores in the target (908). This could include comparing a set ofdetermined pairwise correlation coefficients, independent and/orprincipal component loadings, or other similarity metrics in order todetermine sets of time-varying waveforms of the first plurality oftime-varying waveforms that are similar and that could be associatedwith a single fluorophore. A location of such a single fluorophore couldbe determined based on the locations of the light-sensitive elements ofthe first light sensor that were used to generate such a determined setof time-varying waveforms (e.g., by determining a centroid of thelocations of such light-sensitive elements). A color of such a singlefluorophore could be determined by comparing determined correlationsbetween time-varying waveforms of light detected from the location ofthe single fluorophore by a light-sensitive element of the first lightsensor and time-varying waveforms of light received by light-sensitiveelements of the second light sensor (e.g., by determining a color of thesingle fluorophore based on wavelengths of light corresponding togenerated time-varying waveforms of light having correlations with thetime-varying waveform(s) of light corresponding to the location of thefluorophore that are greater than a threshold). Other methods ofdetermining locations and/or colors of fluorophores in a target based ontime-varying waveforms of light as described herein are anticipated.

The method 900 additionally includes determining, based on thedetermined colors and locations of the two or more fluorophores, anidentity of a probe that is located in the target and that comprises thetwo or more fluorophores (910). This could include matching detectedpatterns of fluorophore colors (e.g., orders of colors of linearly,circularly, or otherwise arranged patterns fluorophores) within theenvironment to known patterns of fluorophores that corresponds to theidentities of potential probes in the environment. This (910) couldinclude using determined colors and locations of fluorophores toidentify and/or locate probes within the target, e.g., by matching apattern of fluorophore locations and/or an order of fluorophore colorsto a template pattern or order that corresponds to a particular type ofprobe. In some examples, identifying a probe could include determining aprobability that a set of detected fluorophores (e.g., fluorophoreswhose locations and/or colors have been determined) correspond to asingle probe and/or determining the probability that such a probe is ofone or more types. This could include determining a number of possibleprobe identifications based on one or more missing fluorophores in adetected pattern of fluorophores in a target. Other methods foridentifying a probe, based on determined locations and colors offluorophores in a target, are anticipated.

The method 900 could include other additional steps or elements. Themethod could include electronically controlling a spatial lightmodulator (SLM) to control a direction, magnitude, or other propertiesof the spectral dispersion applied to the light received by the secondlight sensor. The method 900 could include operating a micromirrordevice to control the angle of actuated mirrors of the micromirrordevice, e.g., to direct light from the particular region of the targettoward one or both of the first and second light sensors. The method 900could include illuminating the particular region of the target, e.g., byreflecting illumination generated by a laser or other light sourcetoward the particular region of the target by controlling one or moreactuated mirrors of a micromirror device to reflect the illuminationtoward the particular region of the target. The method 900 could includeany additional steps, or could include details of implementation of thelisted steps 902, 904, 906, 908, 910 or of other additional steps, asdescribed herein in relation to the operation of an imaging system.Additional and alternative steps of the method 900 are anticipated.

FIG. 10 is a flowchart of an example method 1000 for operating elementsof an imaging system to identify spatially and spectrally multiplexedprobes in a target environment and/or to provide some other functionsand/or applications of the imaging system. The method 1000 includescontrolling a spatial light modulator (SLM) such that at least one ofthe gradient or magnitude of a controllable gradient of the refractiveindex of a refractive layer of the SLM are different during each of aplurality of period of time (1002). In some examples, the controlledrefractive index could be a refractive index of a chromaticallydispersive refractive layer such that light directed toward, reflectedfrom, transmitted through, or otherwise having interacted with the SLMis spectrally dispersed. In some examples, the SLM could further includea reflective layer disposed beneath the refractive layer. In someexamples, the SLM could include an array of regions having respectiveelectronically controllable refractive indexes and electronicallycontrolling the SLM (1002) could include electronically controlling therefractive indexes of the cells such that the refractive indexes of thecells vary in a direction corresponding to a specified direction of thecontrollable gradient at a spatial rate of change corresponding to aspecified magnitude of the controllable gradient.

The method 1000 additionally includes generating, using a light sensor,a plurality of images of the target, wherein each image corresponds to arespective one of the plurality of periods of time (1004). This couldinclude operating a charge-coupled device (CCD), CMOS camera, array ofactive pixel sensors, or other imaging apparatus to generate imagesduring the plurality of periods of time.

The method 1000 further includes determining, based on the plurality ofimages, locations and colors of two or more fluorophores in the target(1006). This could include using a process of deconvolution. Such adeconvolution could be based on information about the operation of theSLM during the plurality of periods of time (e.g., information about thedirection and magnitude of the controllable gradient of the refractiveindex of the refractive layer of the SLM during each of the periods oftime) and based on information about an imaging system used to receivelight from the target, apply the light to the SLM, and provide the lightfrom the SLM to the light sensor. Other methods, such as methodsdescribed in combination with FIGS. 2A-D, could be used to determine thelocations and colors of two or more fluorophores in the target.

The method 1000 additionally includes determining, based on thedetermined colors and locations of the two or more fluorophores, anidentity of a probe that is located in the target and that comprises thetwo or more fluorophores (1008). This could include matching detectedpatterns of fluorophore colors (e.g., orders of colors of linearly,circularly, or otherwise arranged patterns fluorophores) within theenvironment to known patterns of fluorophores that corresponds to theidentities of potential probes in the environment. This (1008) couldinclude using determined colors and locations of fluorophores toidentify and/or locate probes within the target, e.g., by matching apattern of fluorophore locations and/or an order of fluorophore colorsto a template pattern or order that corresponds to a particular type ofprobe. In some examples, identifying a probe could include determining aprobability that a set of detected fluorophores (e.g., fluorophoreswhose locations and/or colors have been determined) correspond to asingle probe and/or determining the probability that such a probe is ofone or more types. This could include determining a number of possibleprobe identifications based on one or more missing fluorophores in adetected pattern of fluorophores in a target. Other methods foridentifying a probe, based on determined locations and colors offluorophores in a target, are anticipated.

The method 1000 could include other additional steps or elements. Themethod 1000 could include operating a micromirror device to control theangle of actuated mirrors of the micromirror device, e.g., to directlight from the target toward the SLM. The method 1000 could includeilluminating the target, e.g., by reflecting illumination generated by alaser or other light source toward one or more specified regions of thetarget by controlling one or more actuated mirrors of a micromirrordevice to reflect the illumination toward the one or more regions of thetarget. The method 1000 could include any additional steps, or couldinclude details of implementation of the listed steps 1002, 1004, 1006,1008 or of other additional steps, as described herein in relation tothe operation of an imaging system. Additional and alternative steps ofthe method 1000 are anticipated.

VII. IMAGING USING A SPECTRALLY DISPERSED ILLUMINATION

The above methods and systems for imaging of a sample (e.g., abiological sample) in order to identify probes in the sample, to detectthe location, color, or other properties of fluorophores in the sample(e.g., fluorophores of such a probe), or to provide some other benefitare intended as non-limiting example embodiments. In some examples, itcould be advantageous to image a sample in such a way that theexcitation and emission spectra of fluorophores (e.g., of probes) orother contents of the sample may be efficiently detected or determined.This could include applying illumination to the sample that is spatiallyencoded with respect to wavelength; that is, illuminating the samplesuch that different regions of the sample receive illumination ofdifferent wavelengths. Embodiments of the present disclosure may beimplemented using a microscope, such as a fluorescence microscope, aconfocal microscope (with confocality along at least one dimension), atransmission microscope, or a reflectance microscope, having one or more2-D imaging devices, e.g., a CCD or CMOS sensor or camera.Alternatively, an optical system may be built according to embodimentsof the present disclosure using suitable optical elements.

Rather than acquiring a hyperspectral image of the sample for eachexcitation wavelength of interest, embodiments of the present disclosureallow for acquiring a 2-D image of emission spectra corresponding tomore than one excitation wavelengths for a subset of areas on a sample.A plurality of the 2-D images can be acquired and computationallyreconstructed to obtain a 4-D hyperspectral-imaging dataset of a sample.

According to an aspect of the present disclosure, excitation lighthaving one or more wavelengths may be used to excite fluorophores in thesample. The excitation light may be generated by a multi-color lightsource that emits light with one or more wavelengths. In someembodiments, the multi-color light source may have a continuousspectrum. For example, the multi-color light source may be a broadbandlight source, such as a supercontinuum laser, a white light source(e.g., a high-pressure mercury lamp, a xenon lamp, a halogen lamp, or ametal halide lamp), or one or more LEDs. In other embodiments, themulti-color light source may have a discrete spectrum. For example, themulti-color light source may be a combination of pulsed or continuous“single-wavelength” lasers that emit light with very narrow spectra.

According to an aspect of the present disclosure, excitation lightemitted by the light source may be structured for exciting a subset ofareas on the sample in an excitation pattern using a spatial lightmodulator (SLM). To structure the excitation light, the SLM may modulatethe phase or amplitude of the excitation light by selectively actuatingor switching its pixels. In some embodiments, the SLM may be selectedfrom a group of SLMs including a digital micromirror device (DMD), adiffractive optical element, a liquid crystal device (LCD), and a liquidcrystal-on-silicon (LCOS) device.

According to an aspect of the present disclosure, the structuredexcitation light may be spectrally dispersed in a first lateraldirection (e.g., the vertical direction y or the horizontal directionx). Spectral dispersion of the excitation light may separate or splitone or more wavelengths of the spectrum of the excitation light in thefirst lateral direction. In some embodiments, at least one dispersiveelement may be used to spectrally disperse the excitation light beforeit illuminates the sample in the excitation pattern. The at least onedispersive element may be a diffractive grating or a prism, or acombination of one or more prisms. Therefore, a spectrally dispersedexcitation pattern may be generated to illuminate areas at variousspatial locations on the sample.

Fluorophores or other types of optical labels in the sample may beexcited by the excitation light illuminating the sample. When they relaxto the ground state, the fluorophores or optical labels may emit lightin a range of wavelengths known as the emission spectrum. Thefluorophores or optical labels may have different emission spectracorresponding to different wavelengths of the excitation light.

As described herein, fluorophores are used in this disclosure as anexemplary optical label. Descriptions in references to fluorophores areequally applicable to other types of optical labels consistent with theembodiments of this disclosure. For example, the excitation lightemitted from the light source may also excite other types of opticallabels, which upon excitation, may emit light with an emission spectrum.Therefore, fluorescent light and fluorescence emission spectrum used inthe descriptions in this disclosure may also be used to represent theemission light and emission spectra of other optical labels.

According to an aspect of the present disclosure, fluorescent lightemitted by the fluorophores excited by the excitation light in a givenarea of the sample may be spectrally dispersed in a second lateraldirection (e.g., the horizontal direction x or the vertical directiony). At least one dispersive element may be employed to spectrallydisperse the fluorescent light into a fluorescence emission spectrumcorresponding to the excitation wavelength at that given area. Thefluorescence emission spectra of a subset of areas on the sample may beacquired as a 2-D image in one exposure by the 2-D imaging device.

According to an aspect of the present disclosure, fluorescenceexcitation and emission spectra of all the areas across the sample oracross a field of view may be acquired by scanning the spectrallydispersed excitation pattern in the first and second lateral directionsand acquiring a 2-D image of the fluorescence emission spectra at eachspatial location of the excitation pattern.

In some embodiments, the excitation pattern is scanned across the sampleor the field of view by modulating the pixels of the SLM. In otherembodiments, an x-y translation stage may be used to laterally scan theexcitation pattern across the sample or the field of view by moving thesample or a diffraction grating in the first and second lateraldirections. The stage may be a motorized translation stage, apiezoelectric translation stage, or any suitable stage that allows forlateral linear movement.

Advantageously, the 4-D hyperspectral-imaging dataset may becomputationally reconstructed from the 2-D images of the emissionspectra, each 2-D image corresponding to the excitation pattern at adifferent spatial location on the sample.

In some aspects, systems and methods according to the present disclosureallows for confocal optical sectioning. This may allow for acquisitionof a hyperspectral-imaging dataset for a plurality of focal planes alongan axial direction of the sample. According to an aspect of the presentdisclosure, a hyperspectral-imaging dataset for a focal plane may beacquired by implementing one or more optical pinholes at a planeconjugate to the selected focal plane. The optical pinholes may be oneor more spatial pinholes, or programmable artificial pinholes formed bypixels of a second SLM.

Advantageously, a degree of confocality may be adjusted as needed bychanging the size and/or separation of the artificial pinholes formed bythe SLM. Additionally, a pinhole pattern may be formed by the SLM byselectively modulating or switching its pixels to match the excitationpattern of the excitation light. The pinhole pattern may advantageouslyallow for confocal imaging of a plurality of areas on the samplesimultaneously illuminated by the excitation pattern. This may increasethe speed and/or throughput of acquiring hyperspectral-imaging datasetsacross the sample at the focal plane comparing to traditional confocalmicroscopes that use sequential point-by-point scanning.

Reference will now be made in detail to embodiments and aspects of thepresent disclosure, examples of which are illustrated in theaccompanying drawings. Where possible, the same reference numbers willbe used throughout the drawings to refer to the same or like parts.

As described herein, to illustrate different wavelengths or frequenciesof light, different densities of dotted texture are used in the attacheddrawings. Higher densities correspond to longer wavelengths or lowerfrequencies of light. Additionally, vertical and horizontal directionsare used as examples for illustrating first and second lateraldirections. Alternatively, the horizontal direction may be the firstlateral direction and the vertical direction may be the second lateraldirection. As described herein, any two suitable different directions ora pair of non-parallel, e.g., orthogonal, directions may be used asfirst and second lateral directions.

Exemplary Schemes for Acquiring a Hyperspectral-Imaging Dataset

FIG. 11 graphically illustrates an exemplary scheme for acquiring ahyperspectral-imaging dataset used by methods and systems of the presentdisclosure. As shown in FIG. 11, excitation light having a discretespectrum is illuminated onto a sample in an excitation pattern 1100.Excitation pattern 1100 may include a 2-D array of excitation spots. Forexample, FIG. 11 illustrates a portion of an exemplary 2-D array ofcircular excitation spots. Additional spots of the array may be locatedabove, below, to the left, and/or to the right of the exemplary array(not shown). As described herein, a suitable size of the array, and asuitable shape, size, and/or separation of the spots may bepredetermined according to the application.

The discrete spectrum of the excitation light includes a plurality ofdiscrete wavelengths or a plurality of narrow spectral bands. Thus, whenthe excitation light is spectrally dispersed by a dispersive elementalong a given lateral direction, excitation pattern 1100 may bespectrally dispersed such that different wavelengths of light aredirected to different locations in the given lateral direction. Forexample, as shown in FIG. 11, excitation pattern 1100 may include aplurality of scanning cells 1110, e.g., a 2-D array of scanning cells1110. When the excitation light is spectrally dispersed along thevertical direction, each scanning cell 1110 may include a plurality ofexcitation spots 1112 a, 1112 b, 1112 c, 1112 d, 1112 e, and 1112 fvertically offset from one another and corresponding to differentexcitation wavelengths of the excitation light generated by the spectraldispersion.

The vertical separation between the excitation spots may or may not beuniform, and may be predetermined by various factors, such as theexcitation wavelengths, the size of the spots, and the amount ofdispersion of the excitation light. The total number of the verticallydispersed excitation spots in scanning cell 1110 may depend on thenumber of discrete wavelengths or narrow spectral bands of theexcitation light.

To generate an excitation spectrum of a given spatial location on thesample, spectrally dispersed excitation pattern 1100 as shown in FIG. 11may be scanned in the vertical direction such that the excitation spotscorresponding to different excitation wavelengths may be shifted to thisgiven spatial location. For example, when excitation pattern 1100 isshifted in the vertical direction, areas in each scanning cell 1110previously illuminated by excitation spots 1112 a, 1112 b, 1112 c, 1112d, 1112 e, and 1112 f can be illuminated by different ones of theseexcitation spots. For instance, by shifting excitation pattern 1100 overone excitation spot, the areas previously illuminated by excitationspots 1112 b are illuminated by excitation spots 1112 a, the areaspreviously illuminated by excitation spots 1112 c are illuminated byexcitation spots 1112 b, the areas previously illuminated by excitationspots 1112 d are illuminated by excitation spots 1112 c, the areaspreviously illuminated by excitation spots 1112 e are illuminated byexcitation spots 1112 d, the areas previously illuminated by excitationspots 1112 f are illuminated by excitation spots 1112 e, and the areaspreviously illuminated by excitation spots 1112 a are illuminated byexcitation spots 1112 f shifted from scanning cells located above (notshown).

Areas within each scanning cell 1110 may be scanned by shiftingspectrally dispersed excitation pattern 1100 in the vertical andhorizontal directions. For example, by shifting excitation pattern 1100over the length of scanning cell 1110 in the vertical direction, a givenarea in scanning cell 1110 can be illuminated by the differentexcitation spots corresponding to the different excitation wavelengthsof the light source. By shifting excitation pattern 1100 verticallyand/or horizontally in a continuous fashion or at predeterminedseparations (e.g., based on the desired vertical and/or horizontalresolution) over the lengths of scanning cell 1110, each given area inscanning cell 1110 can be illuminated by the different excitation spots.

As shown in FIG. 11, the excitation spots of excitation pattern 1100 areseparated in the horizontal direction at a given period. Advantageously,the periodic separation of the excitation spots in the horizontaldirection allows for measuring the fluorescence emission spectra of theexcited areas on the sample. For example, the emitted fluorescent lightfrom a given area illuminated by an excitation spot can be spectrallydispersed in the horizontal direction without overlapping with that ofanother area. Therefore, the fluorescence emission spectra of aplurality of areas simultaneously illuminated by the excitation spotscan be generated and acquired by a 2-D imaging sensor or device.

FIG. 11 shows a 2-D image 1200 of the fluorescence emission spectra withthe excitation wavelengths (λ_(a)) represented in the vertical directionand the emission wavelengths (λ_(b)) represented in the horizontaldirection. FIG. 11 graphically illustrates that, in 2-D image 1200,areas excited by excitation spots 1112 a, 1112 b, 1112 c, 1112 d, 1112e, and 1112 f corresponding to different excitation wavelengths maygenerate different fluorescence emission spectra 1212 a, 1212 b, 1212 c,1212 d, 1212 e, and 1212 f extending in the horizontal direction andcorrespondingly offset from one another in the vertical direction.Therefore, a plurality of fluorescence emission spectra can be acquiredin 2-D image 1200, with each emission spectrum corresponding to anexcited spot of excitation pattern 1100 at a different spatial locationon the sample.

As described above, different areas in each scanning cell 1110 may beilluminated by spatially shifting excitation pattern 1100 laterally inthe vertical and horizontal directions. At each spatial position ofexcitation pattern 1100, fluorescence emission spectra of theilluminated areas can be acquired on 2-D image 1200. Therefore, aplurality of 2-D images 1200 of fluorescence emission spectra may beacquired corresponding to a series of excitation patterns 1100 laterallyshifted from one another.

By combining datasets of the acquired 2-D images 1200, a fluorescenceexcitation-emission matrix (EEM) may be acquired for each pixel orspatial location in the 2-D images 1200. The fluorescence EEM may recordor display fluorescence intensities as a function of a plurality ofexcitation wavelengths and a range of emission wavelengths. Therefore, a4-D hyperspectral-imaging dataset of the sample having both theexcitation and emission spectra may be collected and reconstructed fromthe acquired 2-D images 1200.

FIG. 12 graphically illustrates another exemplary scheme for acquiring ahyperspectral-imaging dataset by methods and systems of the presentdisclosure. As shown in FIG. 12, when the excitation light has acontinuous spectrum and is spectrally dispersed by a dispersive elementalong the vertical direction, a focused spot of the excitation light maybe spectrally dispersed into a continuous excitation line along thevertical direction. In such instances, as shown in FIG. 12, excitationpattern 1100 may be spectrally dispersed into a 2-D array of excitationlines 1114, each representing the range of wavelengths of the excitationlight vertically offset in a continuous fashion.

FIG. 12 illustrates a portion of an exemplary 2-D array of excitationlines 1114, which shows a three-by-three 2-D array. Other excitationlines of the array may be located above, below, to the left, and/or tothe right of the exemplary array (not shown). As described herein, asuitable size of the array, a suitable shape, size, and/or separation ofthe excitation lines may be selected according to a given application.

Areas within each scanning cell 1110 may be similarly scanned asdescribed above by shifting spectrally dispersed excitation pattern 1100in the vertical and horizontal directions. An array of fluorescenceemission spectra 1214 corresponding to the array of excitation lines1114 of excitation pattern 1100 may be similarly acquired on 2-D image1200. Each fluorescence emission spectrum 1214 in 2-D image 1200corresponds to a continuous strip on the sample illuminated by anexcitation line 1114 of excitation pattern 1100.

In the scheme shown in FIG. 12, in some embodiments, when shiftingspectrally dispersed excitation pattern 1100 in the vertical direction,an area excited at a first wavelength along excitation line 1114 maythen be excited at a second wavelength longer or shorter than the firstwavelength after the vertical shifting. Therefore, by shiftingexcitation pattern 1100 vertically in a continuous fashion or overpredetermined separations (e.g., based on the desired verticalresolution), areas illuminated by excitation lines 1114 in each scanningcell 1110 can be illuminated in the different excitation wavelengths ofthe light source. Similar to the scheme shown in FIG. 11, other areas onthe sample in each scanning cell 1110 may be illuminated by thedifferent excitation lines by shifting excitation pattern 1100 in thehorizontal direction.

As described herein, the areas on the sample illuminated by excitationpattern 1100 may be substantially determined by the size and shape ofthe excitation spots or excitation lines of excitation pattern 1100. Thesize and shape of the excitation spots or excitation lines may bedetermined by many factors of the optical system, including the size andshapes of the pixels of the SLM, the magnification of the opticalsystem, and the degree of spectral dispersion of the excitation light.

The spatial separation, horizontal and/or vertical, between excitationspots or lines of excitation pattern 1100 may be predetermined based onvarious factors, such as the excitation wavelengths, the size of thesample, the field of view of the optical system, the desired measurementthroughput, spatial resolution, and/or speed, and the amounts ofspectral dispersion of excitation light and/or emitted fluorescentlight.

For example, the spatial separation between the excitation spots orlines in the vertical direction may be predetermined based on the amountof spectral dispersion of the excitation light such that the excitationspots or lines do not overlap in the vertical direction. The spatialseparation between the excitation spots or lines in the horizontaldirection may be predetermined based on the range of the fluorescenceemission spectra in the horizontal direction such that the fluorescenceemission spectra do not overlap with each other in the horizontaldirection.

In some embodiments, the horizontal and/or vertical periods of an arrayof excitation spots for different wavelengths may be the same. In otherembodiments, the horizontal and/or vertical periods of an array ofexcitation spots for different wavelengths may be different. Differentspatial periods may be convenient for computationally reconstructing the4-D hyperspectral imaging dataset in some cases, for example, where theSLM is placed at a Fourier plane of the sample to generate excitationpattern 1100 as described further below.

Embodiments to be described below in reference to schematicrepresentations of optical systems and/or components are directed tosystems and methods for achieving the above-described schemes foracquiring a 4-D hyperspectral-imaging dataset. The schematicrepresentations are to be understood as not being drawn to scale.

Exemplary Optical Systems and Components

FIG. 13 is a schematic representation of an exemplary hyperspectralimaging system 1300. In some embodiments, system 1300 may be afluorescence microscope, a transmission microscope, or a reflectancemicroscope, or a confocal fluorescence microscope (with confocalityalong at least one dimension). Embodiments of the present disclosure areapplicable to other suitable microscopy techniques for performinghyperspectral imaging.

As shown in FIG. 13, system 1300 may include an illumination system anda detection system, each having a plurality of components. Theillumination system may include a light source 1310, a first SLM 1320 a,at least one lens, e.g., lens 1330 a, and at least one dispersiveelement, e.g., dispersive element 1340 a. The detection system mayinclude a 2-D imaging device 1380, at least one lens, e.g., lens 1330 b,and at least one dispersive element, e.g., dispersive element 1340 b.Depending on its layout, geometry, and/or application, system 1300 mayfurther include a beamsplitter 1350, an objective 1360, a polarizer 1390a, and/or a sample holder 1370 where a sample to be imaged is placed.System 1300 may further include other optical elements, such as mirrors,beam dumps, etc.

As described herein, an optical axis of system 1300 may define a pathalong which the excitation light and emitted fluorescent light from thesample propagate through system 1300.

In the illumination system, as shown in FIG. 13, light source 1310 emitsexcitation light 1402, which is directed to SLM 1320 a. Excitation light1402 may be collimated and/or expanded using one or two lenses (notshown). SLM 1320 a may structure excitation light 1402 throughmodulating the phase or amplitude of excitation light 1402 byselectively actuating or switching its pixels. At least a portion of thepixels of SLM 1320 a reflect excitation light 1402 and direct it alongthe optical axis of system 1300.

As shown in FIG. 13, reflected excitation light 1404 transmits throughlens 1330 a and dispersive element 1340 a. Lens 1330 a may collimatereflected excitation light 1404 along the optical axis. Dispersiveelement 1340 a spectrally disperses reflected excitation light 1404along a first lateral direction. For example, dispersive element 1340 amay cause small wavelength-dependent angular deflection to reflectedexcitation light 1404. Spectrally dispersed excitation light 1406 may bereflected by beamsplitter 1350 and directed towards objective 1360.Objective 1360 then focuses spectrally dispersed excitation light 1406to a sample placed on sample holder 1370.

In the detection system, as shown in FIG. 13, fluorescent light emittedby excited fluorophores in the sample is collected and/or collimated byobjective 1360. Fluorescent light 1408 transmits through beamsplitter1350 and dispersive element 1340 b along the optical axis of system1300. Dispersive element 1340 b spectrally disperses fluorescent light1408 along a second lateral direction as described above. Spectrallydispersed fluorescent light 1410 transmits through lens 1330 b and isacquired by 2-D imaging device 1380. Imaging device 1380 may be placedabout one focal length away from lens 1330 b such that lens 1330 b mayimage and focus the spectrally dispersed fluorescent light 1410 onto a2-D sensor of imaging device 1380.

Other configurations of system 1300 are possible using additionaloptical elements, such as mirrors, lenses, etc., as further describedbelow.

Functions and the working principles of various components of system1300 are described in detail below.

Light Source

As described above, light source 1310 may have a continuous spectrum ora discrete spectrum. Light source 1310 may be a white light source, suchas a supercontinuum laser, or a combination of “single-wavelength”lasers with discrete narrow spectral bands. In some embodiments,excitation light 1402 emitted by light source 1310 may be directedstraight towards SLM 1320 a. In other embodiments, excitation light 1402may be collimated and/or expanded by lenses before being incident on SLM1320 a. Additionally or alternatively, excitation light 1402 may bediffused using a diffuser or a despeckling element to reduce the speckleeffect of coherent illumination.

In some embodiments, light source 1310 may be operably connected to acontroller (not shown) having a processor and a computer-readable mediumthat stores instructions or operational steps. These instructions orsteps, when executed by the processor, modulate the operational statesof light source 1310. For example, the processor may activate ordeactivate light source 1310, modulate the duration of a pulse whenlight source 1310 is a pulsed light source, and/or switch or tune theemission wavelengths of light source 1310.

Spatial Light Modulator for Modulating Excitation Light

As described above, to structure excitation light 1402 for illuminatingthe sample in excitation pattern 1100, SLM 1320 a may modulate theamplitude or phase of excitation light 1402 by selectively modulatingits pixels between operational states.

Amplitude Modulation

In some embodiments, the amplitude of excitation light 1402 may bemodulated by SLM 1320 a. For example, SLM 1320 a may be a digitalmicromirror device (DMD) having an array of multiple micromirrors (notshown). These mirrors may be individually actuated to switch between twooperational positions, an “on” position and an “off” position. When amicromirror is configured to be in the “on” position, excitation light1402 is reflected to propagate along the optical axis as reflectedexcitation light 1404 directed to the sample. When a micromirror isconfigured to be in the “off” position, excitation light 1402 isreflected towards a direction deviated from the optical axis and is notdirected to the sample (not shown). In some embodiments, excitationlight 1402 reflected by the “off” micromirrors may be directed to otheroptical elements, such as a mirror or a beam dump (not shown).

In some embodiments, the micromirrors are of a square shape having alength of its sides ranging from about a few micrometers to about 10 μm.Other shapes and sizes of the micromirrors are also possible and may besuitably used. The DMD is typically capable of changing or alternatingthe “on” and “off” positions of the micromirrors very rapidly.

In some embodiments, a single micromirror of the DMD may be referred toas a single pixel. In other embodiments, a plurality of micromirrors maybe referred to as a single pixel. For example, a group of immediatelyadjacent micromirrors may be referred as a single pixel and may bemodulated or actuated to the same position.

An amplitude modulation pattern may be formed by the micromirrors orpixels of the DMD in the “on” position. The amplitude modulation patternmay be imaged onto the sample as excitation pattern 1100 by lens 1330 aand objective 1360. For example, lens 1330 a is used as a tube lens andcombined with objective 1360 to form an imaging configuration. The DMDis placed at a conjugate plane to the sample or at about one focallength before lens 1330 a. Depending on the focal lengths of lens 1330 aand objective 1360, excitation pattern 1100 may be a magnified orde-magnified image of the amplitude modulation pattern.

In other embodiments, to modulate the amplitude of excitation light1402, SLM 1320 a may be a liquid crystal device (LCD) or a liquidcrystal-on-silicon (LCOS) device. Pixels of SLM 1320 a may create anamplitude modulation pattern by manipulating the polarization of lightincident on the pixels. Similar to the DMD, the LCD or LCOS device maybe placed at a conjugate plane to the sample. Pixels of the LCD or LCOSdevice may be electrically modulated between an “on” state and an “off”state in a pixel-by-pixel fashion. The “on” pixels may rotate theorientation of linearly polarized light by about 90° while the “off”pixels do not perform the rotation. In such instances, a first linearpolarizer (not shown) may be used to linearly polarize excitation light1402. A second linear polarizer or a polarizing beamsplitter (PBS) (notshown) may be used to transmit excitation light 1404 reflected by the“on” pixels and block excitation light 1402 reflected by the “off”pixels.

A disadvantage of modulating the amplitude of excitation light 1402using SLM 1320 a is the loss of light during the modulation. This isbecause most of the pixels of SLM 1320 a are typically in the “off”state. Accordingly, most of excitation light 1402 is steered away fromthe optical axis and would not reach the sample, and thus is lost.Excitation light recycling systems may be used to reduce this loss byredirecting off-optical axis excitation light back to the optical axisas described further below.

Phase Modulation

In some embodiments, the phase of excitation light 1402 may be modulatedby SLM 1320 a. SLM 1320 a may be a reflection type LCD or LCOS device.FIG. 14 is a schematic representation of an exemplary system 1300 usinga LCD or LCOS device as SLM 1320 a. As shown in FIG. 14, the LCD or LCOSdevice may be placed close to a conjugate plane 1322 to the sample. Acustom phase modulation pattern may be formed by the pixels of the LCDor LCOS device. The phase modulation pattern may create an array ofoff-axis lens phase profiles. Wavefront of excitation light 1402 maythen be modulated by the phase modulation pattern and form a preliminaryexcitation pattern (e.g., a diffraction pattern) at conjugate plane1322. This preliminary excitation pattern may be a magnified orde-magnified image of excitation pattern 1100 and may include an arrayof focused spots.

Conjugate plane 1322 may be located a short distance beyond SLM 1320 a.The focal plane of the focused spots of the preliminary excitationpattern may be wavelength dependent. Therefore, different wavelengths ofexcitation light 1402 may not all focus on conjugate plane 1322. In someembodiments, the focal plane for the center wavelength of excitationlight 1402 is approximately at conjugate plane 1322. The preliminaryexcitation pattern formed at or close to conjugate plane 1322 is thenimaged onto the sample as excitation pattern 1100 by lens 1330 a andobjective 1360. Although different wavelengths of excitation pattern1100 in this configuration may have slightly different focal planes,modulating the phase of excitation light 1402 increases the efficiencyof using excitation light 1402 comparing to amplitude modulation.

In other embodiments, the LCD or LCOS device may be placed at anaperture plane, which may be a conjugate plane to the back aperture ofobjective 1360 or a Fourier plane to the sample. For example, oneexemplary configuration of system 1300 may have two tube lenses (notshown) placed between SLM 1320 a and objective 1360. A first tube lensmay be located about one focal length behind SLM 1320 a. A second tubelens may be located about two focal lengths behind the first tube lens.Objective 1360 may be located about one focal length behind the secondtube lens.

The pixels of the LCD or LCOS device may form a custom phase modulationpattern to modulate the wavefront of excitation light 1402. Upon thereflection of excitation light 1402 by the LCD or LCOS device, phases atdifferent locations of the wavefront of the reflected excitation light1404 may be selectively changed according to the phase modulationpattern. In some embodiments, pixels of the LCD or LCOS device may beelectrically modulated between an “on” state and an “off” state in apixel-by-pixel fashion. If pixels of the LCD or LCOS device are in the“on” state, they may change the phase of the reflected light by changingthe optical path length of light traveled in the liquid crystal; and ifthey are in the “off” state, they may not change the phase of thereflected light. This allows the phase modulation pattern formed by thepixels to be digitally customized as needed. In other embodiments,pixels of the LCD or LCOS device may have multiple states or levels ofadjustment (e.g., 256 levels) and may be individually modulated todesired states or levels. Advantageously, increasing the states orlevels of adjustment of the pixels increases the continuity of theadjustment of the phase modulation pattern and thus the adjustment ofthe phase of excitation light 1402.

The phase modulation may render wavelets of reflected excitation light1404 having different directions and/or phases. As reflected excitationlight 1404 propagates along the optical axis, each of the tube lensesand objective 1360 may perform Fourier Transform on the wavefront ofreflected excitation light 1404. A diffraction pattern may then beformed at the focal plane of objective 1360. This diffraction pattern isreferred to herein as excitation pattern 1100 when illuminated on thesample.

In the above-described configuration, because the phase modulationpattern is at or approximately at a Fourier plane to the sample, thephase modulation pattern is the inverse Fourier transform of a desiredexcitation pattern 1100 illuminated on the sample. Because FourierTransform includes a scaling factor proportional to the wavelength oflight, the spatial periods of the array of excitation spots fordifferent wavelengths in excitation pattern 1100 may be different. Forexample, longer wavelength would diffract at larger angles, which can beconverted to larger spatial periods. This may cause the correspondingfluorescence emission spectra arrays acquired in 2-D image 1200 to havedifferent spatial periods. Customized computer algorithms may be usedfor generating time-varying phase modulation patterns for scanningacross the field of view and/or for computationally reconstructing the4-D hyperspectral-imaging dataset from datasets of such 2-D images.

Advantageously, modulating the phase of excitation light 1402 may allowit to propagate with substantially uniform intensity in the near fieldof the LCD or LCOS device and thus reduce loss of light. The modulatedexcitation light may then form customizable or programmable excitationpattern 1100 on the sample in the far field. Therefore, comparing tomodulating the amplitude of excitation light 1402 as described above,modulating the phase of excitation light 1402 to create excitationpattern 1100 may substantially increase the efficiency of illuminationof system 1300 by reducing loss of excitation light 1402. Additionally,increasing the continuity of the phase modulation of excitation light1402 may further increase the diffraction efficiency of the LCD or LCOSdevice and thus the efficiency of illumination of system 1300.

The LCD or LCOS device for modulating the amplitude or phase ofexcitation light 1402 may alternatively be a transmission type deviceimplemented along the optical axis. The geometry of the illuminationsystem may be suitably designed such that the amplitude or phasemodulation pattern formed by the pixels of the device may modulate theamplitude or phase of excitation light 1402 similarly as describedabove.

Whether SLM 1320 a modulates the amplitude or phase of excitation light1402, excitation pattern 1100 can be programmed and customized as neededby modulating pixels of SLM 1320 a between two or multiple operationalstates or levels in a pixel-by-pixel fashion. Further, excitationpattern 1100 can be translated or shifted in a given spatial direction,such as the horizontal or vertical direction, by scanning or shiftingthe modulation of the pixels of SLM 1320 a. This advantageously allowsfor scanning excitation pattern 1100 across the field of view of system1300 without moving the sample and/or sample holder 1370 using an x-ytranslation stage.

In some embodiments, depending on the type and modulation features ofthe pixels of SLM 1320 a, excitation light 1402 may be directed towardsSLM 1320 a at a predetermined angle relative to a plane of SLM 1320 a.The predetermined angle may depend on the type of SLM 1320 a and/or thegeometry of system 1300. In some instances, when SLM 1320 a is areflection type SLM that modulates the phase of excitation light 1402,excitation light 1402 may be directed towards SLM 1320 a at an anglesuch that reflected excitation light 1404 propagates along the opticalaxis of system 1300. In other instances, when SLM 320 a is a DMD,excitation light 1402 may be directed towards the DMD at an angle suchthat excitation light 404 reflected by the “on” micromirrors propagatesalong the optical axis.

In some embodiments, SLM 1320 a may be operably connected to acontroller (not shown) having a processor and a computer-readable mediumthat stores instructions or operational steps. These instructions orsteps, when executed by the processor, modulate the operational statesof the pixels of SLM 1320 a to form a desired excitation pattern 1100and/or to translate excitation pattern 1100 in a desired spatialdirection over a predetermined distance across the field of view.

Lenses and Objective

Various lenses of system 1300, such as lenses 1330 a and 1330 b, may beachromatic, such as achromatic doublets or triplets, to limit or reducethe effects of chromatic and/or spherical aberration of the system.Further, objective 1360 of system 1300 may be achromatic. Alternativelyor additionally, objective 1360 may be an infinity-corrected objectivesuch that objective 1360 may form a desired focus (e.g., focused spotsor focused pattern) of a collimated light beam entering from its backaperture. Using achromatic lenses and/or achromatic orinfinity-corrected objective may allow excitation light 1402 ofdifferent wavelengths to have at least approximately the same focus inthe sample. Further, using achromatic lenses and/or achromatic objectivemay allow fluorescent light of different wavelengths from a focal planein the sample to similarly form a focused image at imaging device 1380.Therefore, using achromatic lenses and/or achromatic objective mayimprove the quality of 2-D images 1200 of fluorescence emission spectra,and thus the quality of the reconstructed hyperspectral-imaging dataset.

Dispersive Elements

Dispersive elements 1340 a and 1340 b may be diffraction gratings orprisms, such as non-deviating prisms (e.g., Amici prisms or double Amiciprisms). The types of dispersive elements 1340 a and 1340 b may be thesame or may be different. The degree of dispersion caused by dispersiveelements 1340 a and 1340 b may be same or different, and may bepredetermined based on various factors, such as the spectral ranges ofexcitation light and fluorescent light, the size of the sample or thefield of view, the size of imaging device 1380, the desired spectralresolution, and the application of system 1300.

In some embodiments, the degree of dispersion caused by dispersiveelements 1340 a and 1340 b may be adjustable. For example, dispersiveelement 1340 a may be a pair of double Amici prisms placed along theoptical axis of system 1300. At least one of the pair of double Amiciprisms is rotatable relative to the other around the optical axis. Therotation of the double Amici prisms relative to each other may allow forcontinuous control of the amount and/or angular orientation of thespectral dispersion of excitation light 1402. Similarly, dispersiveelement 1340 b may be a pair of double Amici prisms, allowing forcontinuous control of the amount and/or angular orientations of thespectral dispersion (e.g., dispersion angles) of fluorescent light 1408.

Excitation Light Blocking

Because the intensity of excitation light 1402 may be orders ofmagnitude stronger than fluorescent light 1408, excitation light 1402reflected and/or scattered by the sample and/or sample holder 1370 mayenter the detection system and affect the detection or acquisition ofthe fluorescence emission spectra by imaging device 1380. Therefore,embodiments of the present disclosure may reduce or block excitationlight 1402 from propagating into the detection system as describedbelow.

In some embodiments, beamsplitter 1350 may be used to reject or blockexcitation light 1402 from propagating into the detection system. Forexample, beamsplitter 1350 of system 1300 may be a dichroicbeamsplitter, a polarizing beamsplitter (PBS), or other suitable type ofbeamsplitter.

When light source 1310 or excitation light 1402 has a discrete spectrumhaving one or more discrete wavelengths or narrow spectral bands,beamsplitter 1350 may be a dichroic beamsplitter that selectivelyreflects and transmits light depending on its wavelength. For example,beamsplitter 1350 may be a multiband dichroic that has multiple cut-offwavelengths and passbands. The multiband dichroic may be selected tosubstantially reflect wavelengths of excitation light 1402 and tosubstantially transmit wavelengths of fluorescent light 1408. In suchinstances, some wavelengths of fluorescent light 1408 that are the sameor close to that of excitation light 1402 may be substantially blocked,and thus may have substantially reduced intensity in 2-D image 1200acquired by imaging device 1380.

Alternatively or additionally, when light source 1310 or excitationlight 1402 has a discrete spectrum, a set of corresponding notch filtersor a single multi-notch filter (not shown) may be added to the detectionsystem along the optical axis. The notch filters or multi-north filtermay selectively reflect the discrete wavelengths or narrow spectralbands of excitation light 1402, thereby blocking excitation light 1402from reaching imaging device 1380. Again, some wavelengths offluorescent light 1408 that are the same or close to that of excitationlight 1402 may be substantially blocked by the notch filters, and thusmay have substantially reduced intensity in 2-D image 1200 acquired byimaging device 1380.

When light source 1310 or excitation light 1402 has a continuousspectrum, beamsplitter 1350 may be a long-pass dichroic beamsplitterthat reflects at least a portion of the wavelengths of excitation light1402 and transmits at least a portion of the wavelengths of fluorescentlight 1408. The spectrum of excitation light 1402 typically ranges fromthe ultraviolet through the visible spectra, and the spectrum offluorescent light 1408 typically ranges from the visible into the nearinfrared spectra. Therefore, the long-pass dichroic beamsplitter mayblock wavelengths of excitation light 1402 and transmit wavelengths offluorescent light 1408. However, in some instances, both the spectrum ofexcitation light 1402 and spectrum of fluorescent light 1408 may includeshort to long wavelengths and they may overlap, e.g., in the visiblespectrum. In such instances, the long-pass dichroic beamsplitter mayblock at least some fluorescent light 1408 in the visible spectrum, andmay not be suitable for rejecting excitation light 1402 in applicationswhere the blocked spectrum of fluorescence light 1408 contains desiredspectral information, for example.

Regardless of the types of spectrum of light source 1310 or excitationlight 1402 (whether or not discrete or continuous), in some embodiments,polarizer 1390 a and beamsplitter 1350 may be used in combination toblock excitation light 1402 from entering the detection system and thusfrom propagating towards imaging device 1380. For example, beamsplitter350 may be a polarizing beamsplitter (PBS) that reflects light whosevibration orientation aligns with the transmission axis of polarizer1390 a.

For example, polarizer 1390 a may be placed at any suitable locationalong the optical axis to linearly polarize excitation light 1402. ThePBS may be selected to reflect light having a vibration orientation sameas that of the polarized excitation light and to transmit light having avibration orientation perpendicular to that of the polarized excitationlight. Most of the excitation light collected by objective 1360 wouldtherefore reflect from this PBS and would not reach imaging device 1380.In some instances, both the sample and objective 1360 may depolarizereflected and/or scattered excitation light to a small degree, and thusundesirably allow some excitation light to transmit through the PBS andenter the detection system.

2-D Imaging Device

Imaging device 1380 may include a suitable 2-D sensor located at animage plane conjugate to a selected focal plane in the sample. Thesensor could be implemented with a CMOS sensor, a CCD sensor, a 2-Darray of silicon avalanche photodiodes (APDs), an electron-multipliedCCD (EMCCD), an intensified CCD, or other suitable types of 2-D sensors.

Imaging device 1380 may be operatively connected to a controller or acomputing device (not shown) that controls its operation. For example,controller (not shown) may have a processor and one or morecomputer-readable medium that stores instructions or operational steps.The instructions or operational steps, when executed by the processor,may operate the exposure of imaging device 1380, acquire 2-D images1200, and/or store the datasets of 2-D image 1200 to a memory. Thecomputer-readable medium may further store instructions or operationalsteps that, when executed by the processor, perform data processing ofthe acquired 2-D image datasets and/or reconstruct the 4-Dhyperspectral-imaging dataset from the 2-D image datasets.

System 1300 may advantageously have additional technical features andcapabilities to enhance its functionality and performance as describedin detail below.

Time-Resolved Capability

In some embodiments, time-resolved capability may be advantageouslyadded to system 1300 to allow for fluorescence lifetime imaging (FLIM)or time-resolved fluorescence spectroscopy. For example, a pulsed lightsource, such as a supercontinuum laser, may be used as light source1310, together with a 2-D imaging device 1380 having picosecond tonanosecond time-gating capability, such as an intensified CCD camera oran optoelectronic streak camera. Alternatively, a conventional 2-D CCDor CMOS sensor may be used in combination with an electro-optic shutter.In some embodiments, a modulated, electron-multiplied fluorescencelifetime imaging microscope (MEM-FLIM) camera may be used in combinationwith a modulated light source 1310, e.g., a pulsed light source.

The lifetime of the fluorophores or fluorescent molecules in the samplemay be calculated from the acquired time-resolved 2-D images of thefluorescence emission spectra for each spatial location in the field ofview. This adds another dimension of information to thehyperspectral-imaging dataset, thereby providing additional informationabout the fluorophores or fluorescent molecules in the sample.

Because FLIM excites the fluorophores with short excitation pulses inthe time-domain, the FLIM capability of system 1300 may substantiallyreject the scattered and/or reflected excitation light by discarding thesignals close to zero delay. This may advantageously reduce or minimizethe effect of the scattered and/or reflected excitation light in theacquired fluorescence signals, e.g., 2-D image 1200.

Fluorescence Polarization

In some embodiments, system 1300 may advantageously allow forfluorescence polarization (or anisotropy) measurement to obtainadditional information about the fluorophores or fluorescent moleculesin the sample. Relationships between the polarization of the excitationlight and the emitted fluorescent light subsequently detected may beused to analyze and study various chemical and/or physical processes ofthe molecules in the sample, such as rotational diffusion, bindinginteractions, and orientation.

To add the capability for measuring fluorescence polarization, as shownin FIG. 15, system 1300 may include polarizer 1390 a and an opticalelement, such as a waveplate or a polarizer. For example, the opticalelement may be an achromatic half-wave plate (HWP) 1390 b. Polarizer1390 a may be a linear polarizer located in the illumination system,thereby generating linearly polarized excitation light, e.g., verticallypolarized light. Depending upon the orientation of their absorptiondipoles, individual fluorophores in the sample are preferentiallyexcited by the linearly polarized excitation light. The fluorescentlight emitted from the sample may be partially depolarized due to randomorientation, diffusion, and/or rotation of the fluorophores.

Beamsplitter 1350 may be a polarizing beamsplitter (PBS) thatsubstantially transmits horizontally polarized light and reflectsvertically polarized light. For example, as shown in FIG. 15, theexcitation light vertically polarized by polarizer 1390 a can bereflected by the PBS and then propagates towards HWP 1390 b. HWP 1390 bmay be placed at a suitable location, such as before beamsplitter 1350along the optical axis. In such instances, HWP 1390 b may rotate thevibration orientations of both the linearly polarized excitation lightand the collected fluorescent light from the sample by about twice theangle between a vertical axis and the fast axis of the HWP, for example.Rotating HWP 1390 b around the optical axis would advantageously rotatethe vibration directions of both the excitation light and fluorescentlight. Beamsplitter 1350 may substantially block the polarizedexcitation light and transmit at least a portion of polarizedfluorescent light to be acquired by imaging device 1380.

Depending on the application, such fluorescence polarization assays maybe performed in steady state or with time-resolved measurements, such asutilizing the FLIM capability as described above.

Measurement of fluorescence polarization (or anisotropy) adds anotherdimension of information to the hyperspectral-imaging dataset acquiredby system 1300. This additional dimension of information may complementthe information in the other dimensions of the hyperspectral-imagingdataset about the local chemical and physical environments offluorophore-tagged molecules in the sample, such as molecular mass andorientation of the molecules. The augmented hyperspectral-imagingdataset acquired by system 1300 may further improve the accuracy ofdiagnosis of physiologic or pathologic changes of the sample.

Excitation Light Recycling System

As described above, because most of excitation light 1402 is steeredaway from the optical axis and would not reach the sample, modulatingthe amplitude of excitation light 1402 using SLM 1320 a, e.g., a DMD ora LCD, to generate excitation pattern 1100 results in loss of light.Therefore, in some embodiments, system 1300 may advantageously includean excitation light recycling system 1500 to increase efficiency ofutilization of excitation light 1402. Recycling system 1500 may redirectthe off-optical axis excitation light back to the optical axis towardsthe sample as described below.

Reflection-Based Scheme

In some embodiments, excitation light recycling system 1500 uses areflection-based scheme as shown in FIG. 16. Recycling system 1500 mayinclude one or more lenses and mirrors. For example, recycling system1500 may include a lens 1330 c, a flat mirror 1510, a first concavemirror 1520 a, and a second concave mirror 1520 b. The concave mirrorsmay be replaced by a combination of a flat mirror and a lens.

Excitation light 1402 may be collimated before passing lens 1330 c. Lens1330 c may focus collimated excitation light 1402 to a focal point 1312a of mirror 1520 a at a given plane 1312 near or aligned with the planeof SLM 1320 a. Then, excitation light 1402 expands as it propagates fromfocal point 1312 a to mirror 1520 a. Mirror 1520 a may re-collimateexcitation light 1402 and direct it to SLM 1320 a.

As described above, when SLM 1320 a is a DMD, a small fraction ofexcitation light 1402 may be reflected by the “on” pixels towards lens1330 a along the optical axis, while the rest, e.g., off-axis excitationlight 1403, is reflected by the “off” pixels at a different angle andaway from the optical axis. Mirror 1520 b may be configured to interceptthis off-axis excitation light 1403 and reflect it back to a point 1312b very close to focal point 1312 a. The separation between point 1312 band focal point 1312 a may be just large enough to allow the edge ofmirror 1510 to intercept the returned off-axis excitation light 1403without substantially blocking the original excitation light 1402.Mirror 1510 then may direct off-axis excitation light 1403 back to SLM1320 a via a path that is nearly aligned with the original path. In suchconfiguration, excitation light 1402 can be incident onto SLM 1320 amany times through multiple reflections between the mirrors, therebyrecycling off-axis excitation light 1403 back to the optical axis.

As described herein, the three paths for the recycling of off-axisexcitation light 1403 shown in FIG. 16 are exemplary only. Multiple orinfinite recycling paths may be possible.

A few design considerations of system 1300 are discussed in thefollowing. In some instances, the recycled off-axis excitation light1403 may be slightly divergent. For each recycling path of off-axisexcitation light 1403 propagating in recycling system 1500, becauseoff-axis excitation light 1403 is not returned to focal point 1312 a,off-axis excitation light 1403 would have a slightly different anglewhen it reaches SLM 1320 a from that of the original excitation light1402. The angular difference (or divergent angle) may be defined asΔθ=Δx/f, where “Δx” is the separation between focal point 1312 a andpoint 1312 b, and “f” is the focal length of mirror 1520 a (or a lens)for re-collimating the off-axis excitation light reflected by mirror1510. Δx may be at least greater than any unusable rough edge of mirror1510, and greater than the diffraction limited spot size of excitationlight 1402. Depending on the values of Δx and f, Δθ may be less than 1degree. Such small degree of angular difference (or divergence angle)may not affect the formation of excitation pattern 1100.

In some instances, when SLM 1320 a is a DMD, the DMD may have adiffraction effect on reflected excitation light 1404. For example, asingle micromirror of the DMD may have a side length of approximately 10μm. A typical divergence angle for reflected excitation light 1404caused by the diffraction of the micromirror array may be about λ_(a)/10μm, where λ_(a) is the wavelength of excitation light 1402. Therefore,the divergence angle may be about less than one radian, e.g., 1/20radian, or less than a few degrees, e.g., 3 degrees. Thus, most ofexcitation light 1404 reflected by the “on” pixels or micromirrors ofthe DMD from different recycling paths in recycling system 1500 mayoverlap and propagate along the optical axis, and thus may not affectthe formation of excitation pattern 1100.

In some instances, reflected excitation light 1404 from differentrecycling paths in recycling system 1500 may exhibit opticalinterference. For a light source 1310 having discrete wavelengths ornarrow spectral bands, this interference may cause reflected excitationlight 1404 to have unstable intensities when focused on the sample.Additional optical components may be added to control the relativephases of excitation light 1403 propagating in different recycling pathsto reduce the optical interference effect. However, this may complicatethe design of system 1300. Therefore, the reflection-based scheme shownin FIG. 16 for recycling system 1500 may be more suitable for systems1300 having a light source 1310 with a broadband spectrum, such as awhite light source. For such systems 1300, the effect of theinterference may impose very rapid small oscillations on the spectrum ofreflected excitation light 1404. Fluorophores typically have spectrallybroad absorption features, allowing these oscillations to average outduring excitation. Therefore, the optical interference effect may havelittle effect on the acquired fluorescence emission spectra when lightsource 1310 has a broadband spectrum.

Polarization-Based Scheme

To solve the above-described technical problem for recycling excitationlight 1402 having discrete wavelengths or narrow spectra bands, in someembodiments, excitation light recycling system 1500 may use apolarization-based scheme as shown in FIG. 17.

As shown in FIG. 17, polarization-based recycling system 1500 mayinclude one or more optical components. For example, recycling system1500 may include an optical isolator 1530, a polarizing beamsplitter(PBS) 1540, a quarter-wave plate (QWP) 1550, and one or more mirrors,e.g., a first mirror 1510 a and a second mirror 1510 b. In someembodiments, optical isolator 1530 may include a linear polarizer or maybe optionally replaced by a linear polarizer.

In this scheme, optical isolator 1530 allows the propagation ofexcitation light 402 in only one forward direction. Excitation light1402 may be a linearly polarized light, or may become linearly polarizedafter passing through optical isolator 1530. The linearly polarizedexcitation light after passing through optical isolator 1530 is referredto as excitation light 1420. PBS 1540 may be configured to transmitlight having a vibration orientation parallel with that of excitationlight 1420 and reflect light having a vibration orientation orthogonalto that of excitation light 1420. For example, excitation light 1420 maybe horizontally polarized or have a vibration orientation in ahorizontal direction. PBS 1540 may transmit horizontally polarized lightand reflect vertically polarized light. Therefore, excitation light 1420transmits through PBS 540 and propagates towards SLM 1320 a.

Description below of the polarization-based scheme of recycling system1500 uses excitation light 1420 that is horizontally polarized as anexample. Embodiments of the polarization-based scheme of recyclingsystem 1500 is equally applicable for linearly polarized excitationlight 1420 having any vibration orientation.

As described above, when SLM 1320 a is a DMD, a small fraction ofexcitation light 1420 may be reflected by the “on” micromirrors of theDMD towards lens 1330 a along the optical axis, while the off-axisexcitation light 1403 reflected by the “off” pixels are steered awayfrom the optical axis. Mirror 1510 a may be configured to intercept theoff-axis excitation light 1403 and reflect it back to the “off” pixelson the DMD. Off-axis excitation light 1403 may pass through QWP 550 afirst time when it propagates towards mirror 1510 a and a second timewhen it is directed back to the DMD by mirror 1510 a, which rotate thevibration orientation of off-axis excitation light 1403 by 90°. Forexample, horizontally polarized excitation light 1403 may be changed tobe vertically polarized after passing through QWP 1550 twice. Thevertically polarized excitation light is then reflected by the “off”micromirrors of the DMD towards PBS 1540.

Because the vertically polarized excitation light reflected to PBS 1540has a vibration orientation orthogonal to that of horizontally polarizedexcitation light 1420, it is reflected by PBS 1540 and directed tomirror 1510 b. Without changing its vibration orientation, mirror 1510 band PBS 1540 then reflect the vertically polarized excitation light backto the DMD, where the “on” micromirrors then reflect the verticallypolarized excitation light towards lens 1330 a along the optical axis.The “off” micromirrors reflect the vertically polarized excitationlight, which again transmits through QWP 1550 and mirror 1510 a twiceand becomes horizontally polarized. This horizontally polarizedexcitation light would pass through PBS 1540, but would not propagateback to light source 1310 because of optical isolator 1530.

In the above-described polarization-based scheme of recycling system1500, because QWP 1550 rotates the vibration orientation of off-axisexcitation light 1403 by 90°, excitation light 1404 reflected towardsthe optical axis, which includes the portion of the off-axis excitationlight 1403 that is recycled, would have orthogonal polarizations. Insuch instances, rather than a polarizing beamsplitter, beamsplitter 1350may suitably be a multiband dichroic that has multiple cut-offwavelengths and passbands. As described above, the multiband dichroicmay be selected such that wavelengths of excitation light 1402 having adiscrete spectrum are substantially reflected and wavelengths of emittedfluorescent light 1408 are substantially transmitted. Therefore, thispolarization-based scheme may work better in systems 1300 using a lightsource 1310 having discrete wavelengths or narrow spectra bands, such asa combination of a set of lasers operating at discrete wavelengths.

Confocal Optical Sectioning Capability

As described above, system 1300 may allow for confocal opticalsectioning, which allows for selecting the depth of a focal plane in thesample. The depth of the focal plane may be selected by introducing oneor more optical pinholes at a plane conjugate to the selected focalplane.

FIG. 18 is a schematic representation of an exemplary system 1300 thatallows for confocal optical sectioning. As shown in FIG. 18, system 1300may include the first SLM 1320 a for generating excitation pattern 1100as described above, a second SLM 1320 b for confocal optical sectioning,at least one additional mirror 1510 c, one or more tube lenses, e.g.,1330 d and 1330 e, and a z-axis translation stage or a tunable liquidlens (not shown). SLM 1320 b may have similar types and features asdescribed above for SLM 1320 a. For example, pixels of SLM 1320 b may beindividually modulated in the same manners as those described for SLM1320 a.

SLM 1320 b may be placed at about a plane conjugate to a focal planelocated at a desired depth in the sample along the optical axis. Forexample, lens 1330 b and objective 1360 may form an imagingconfiguration. As shown in FIG. 18, lens 1330 b may be located behindobjective 1360 and SLM 1320 b may be located about one focal lengthbehind lens 1330 b. The space between the back aperture of objective1360 and lens 1330 b is a collimated space, which may be adjusted asneed based on various factors, such as the geometry of system 1300 and adesired location of a minimum beam aperture. In some embodiments, lens1330 b is placed about one focal length behind objective 1360.

Pixels of SLM 1320 b may be selectively actuated or switched to “on” or“off” states to form a pinhole pattern matching or conjugatingexcitation pattern 1100 on the sample. The pinhole pattern may include aplurality of artificial optical pinholes at the conjugate plane andreject out-of-focus fluorescent light from the sample. Therefore,out-of-focus fluorescent light would not pass through the detectionsystem and are substantially removed or eliminated from the acquired 2-Dimage 1200.

The size and separations of the artificial pinholes in the pinholepattern are programmable, and may be customized based on themagnification of the imaging configuration formed by objective 1360 andlens 1330 b. In some instances, the pinhole pattern may include aplurality of “on” pixels in elongated shapes to allow fluorescent lightemitted from multiple locations on the sample (e.g., areas excited byexcitation spots 1112 a-1112 f) to be acquired simultaneously. In otherinstances, the pinhole pattern may include an array of “on” pixels thatmatch the size of the excitation lines or excitation spots in excitationpattern 1100.

The fluorescent light 1412 reflected by the “on” pixels of SLM 1320 b isthen imaged to imaging device 1380 by tube lenses 1330 d and 1330 e. Forexample, mirror 1510 c may be placed at a suitable position along theoptical axis and for directing fluorescent light 1412 reflected by the“on” pixels to the tube lenses. Tube lens 1330 d may be located aboutone focal length beyond the image produced by lens 1330 b (e.g., aboutone focal length behind SLM 1320 b) such that it re-collimates thefluorescent light from the sample. Imaging device 1380 may be locatedabout one focal length behind tube lens 1330 e or at a conjugate planeof SLM 1320 b. Because the fluorescent light is collimated in the spacebetween tube lenses 1330 d and 1330 e, the distance between tube lenses1330 d and 1330 e may be adjusted as desired. In some embodiments, tubelens 1330 e may be about two focal lengths behind tube lens 1330 d suchthat a plane midway between tube lens 1330 d and 1330 e is conjugate toan exit pupil of system 1300.

By digitally changing and/or laterally shifting excitation pattern 1100and the matching pinhole pattern on SLM 1320 b correspondingly, thewhole field of view may be scanned for acquiring a confocal-imagingdataset. By further scanning the field of view across the sample, thewhole sample can be scanned to obtain a complete confocal-imagingdataset of the sample.

In some embodiments, imaging device 1380 may be suitably tilted toreduce aberrations and thus improve the quality of the acquired 2-Dimage dataset. This is at least because the “on” pixels of SLM 1320 bdirect fluorescent light 1412 at an angle that is not perpendicular tothe surface plane of SLM 1320 b such that an image plane formed by tubelenses 1330 d and 1330 e may be tilted. Aberrations caused by thistilting effect may be compensated by properly tilting imaging device1380. Aberrations may be further reduced if a dispersion angle ofdispersive element 1340 b is adjusted to be parallel to a rotation axisof the tilted imaging device 1380.

To change or select a depth of the focal plane, in some embodiments,sample holder 1370 may be installed on the z-axis translation stage. Thedesired depth of the focal plane may be selected by moving sample holder1370 along the optical axis using the z-axis translation stage.Alternatively, objective 1360 may be installed on the z-axis translationstage and the desired depth of the focal plane may be selected by movingobjective 1360 along the optical axis. As describe herein, the z-axistranslation stage may also include x-y translation capability to movethe field of view of system 1300 across the sample in lateraldirections. In other embodiments, the desired depth of the focal planemay be selected by tuning the focus of a tunable liquid lens (not shown)placed behind objective 1360. Additionally, the z-translation stage orthe tunable liquid lens may be controlled by a computer program toachieve autofocusing.

Advantageously, a degree of confocality may be adjusted as needed bychanging the size and/or separation of the artificial pinholes formed bySLM 1320 b. For example, increasing the sizes of the pinholes byincreasing the number of pixels in the pinholes and/or reducing thepinhole spacing may reduce the degree of confocality and thus the degreeof depth selectivity of the desired focal plane. On the other hand,decreasing the size of the pinholes by reducing the number of pixels inthe pinholes and/or increasing the pinhole spacing may increase thedegree of confocality and thus the degree of depth selectivity of thedesired focal plane. In some embodiments, the depth selectivity may beproportional to the ratio of the number of “off” and “on” pixels of SLM1320 b. Therefore, SLM 1320 b may advantageously allow for switchingbetween wide-field and confocal imaging as desired by convenientlyadjusting the pinhole size and/or separation.

Additionally, the pinhole pattern formed by pixels of SLM 1320 badvantageously allows for confocal imaging of a plurality of areas onthe sample simultaneously illuminated by excitation pattern 1100. Thismay increase the speed and/or throughput of acquiringhyperspectral-imaging datasets across the sample at the desired focalplane comparing to traditional confocal microscopes that use sequentialpoint-by-point scanning.

As shown in FIG. 18, in embodiments of system 1300 using SLM 1320 b,dispersive element 1340 b may be located in the collimated space betweentube lenses 1330 d and 330 e. Because the pinhole pattern on SLM 1320 bmatches excitation pattern 1100 on the sample, fluorescent light 1412reflected by the artificial pinholes of SLM 1320 b can be dispersed bydispersive element 1340 b as described above such that the fluorescenceemission spectra corresponding to the excitation spots of excitationpattern 1100 can be acquired by the 2-D sensor of imaging device 1380.

Selective Filtering of Fluorescence Emission Spectrum

In some applications, different fluorophores having fluorescenceemission spectra that are spaced apart, such as green and redfluorophores, may be used or exist in the sample. This may result inlateral gaps in a fluorescence emission spectrum acquired in 2-D image1200 along the emission wavelength axis, resulting in inefficient use ofthe space on the 2-D sensor of imaging device 1380.

In other applications, the combination of different fluorophores mayresult in an overall broad fluorescence emission spectrum to be acquiredby imaging device 1380. In some instances, multiple spectral regionswithin the broad emission fluorescence spectrum may be more useful thanother regions. Acquiring the complete broad fluorescence emissionspectrum may result in inefficient use of the space on the 2-D sensor ofimaging device 1380 and further reduce the throughput of acquiring thehyperspectral-imaging dataset.

To increase the efficiency of using the sensor space of imaging device1380 and increase the throughput of system 1300, a spectral slicingsystem 1342 may be included at a collimated space along the optical axisin the detection system. For example, as shown in FIG. 18, spectralslicing system 1342 may be located between tube lenses 1330 d and 1330e, and may be placed before dispersive element 1340 b in the detectionsystem. Spectral slicing system 1342 may selectively pass one or morespectral bands with tunable bandwidths and/or center wavelengths,thereby allowing for acquiring a hyperspectral-imaging dataset withdesired spectral bands and/or desired spectral resolutions.

As shown in FIG. 18, spectral slicing system 1342 may include aplurality of spectral slicing modules 1344. Fluorescent light 1408emitted by the sample may enter spectral slicing system 1342 after beingcollimated or re-collimated. Using one or more beamsplitters and/ormotorized flip mirrors, spectral slicing system 1342 may split an inputcollimated fluorescent light beam into one or more beams, each having adifferent spectral band, and direct them through spectral slicingmodules 1344 respectively. Each spectral slicing module 1344 may filterone of the beams to have a desired bandwidth and a center wavelength.After the filtering, spectral slicing system 1342 may combine thefiltered beams into an output beam using one or more beamsplittersand/or motorized flip mirrors.

Spectral slicing modules 1344 may each operate as a tunable bandpassfilter with a tunable passband width and/or a tunable center wavelength.For example, spectral slicing module 1344 may include a long-pass filterand a short-pass filter along its optical axis. At least one of thelong-pass filter and short-pass filter is rotatable relative to theoptical axis. Rotating the filters may adjust the angle of incidence ofthe beam on the filters and thus shift the wavelengths of theirabsorption or reflection edges. Thus, rotating the long-pass filterand/or short-pass may tune the bandwidth and/or center wavelength of thespectral passband formed by the long-pass and shot-pass filters.Alternatively, spectral slicing modules 1344 may each include a tunablebandpass filter whose passband may be tuned by rotating the filter andthus tuning the angle of incidence of the beam on the filter.

Spectral slicing system 1342 allows the measured fluorescence emissionspectra to be adjustably filtered to desired spectral ranges useful fora particular application. By selecting the desired spectral ranges, thespace on the 2-D sensor of imaging device 1380 can be used moreefficiently. For example, as described above, the degree of dispersioncaused by dispersive element 1340 b can be adjustable. The spectralresolution of the selected spectral ranges of the fluorescence emissionspectra may be increased by increasing the degree of spectral dispersionusing dispersive element 1340 b, thereby providing more information ofthe fluorophores or fluorescent molecules in the sample.

Additionally, selecting the desired spectral ranges may allow forreducing the lateral spacing between the fluorescence emission spectrain 2-D image 1200 along the emission wavelength axis, thereby improvingthe throughput of dataset acquisition. For example, by reducing theperiod of excitation pattern 1100 in the horizontal direction, anddecreasing the degree of spectral dispersion using dispersive element1340 b, the period of the array of fluorescence emission spectra in thehorizontal direction in 2-D image 1200 may be reduced. This may in turnincrease the number of fluorescence emission spectra that can beacquired in one exposure, thereby increasing the efficiency of using thesenor space of imaging device 1380.

Alternative Configurations

In some applications, more compact configurations of system 1300 may bedesirable. In such instances, system 1300 may use diffractive elementsin place of SLM 1320 a and/or SLM 1320 b. Embodiments of suchconfigurations of system 1300 are described below in reference to FIGS.19-23.

FIG. 19 is a schematic representation of an exemplary compact embodimentof system 1300. As shown in FIG. 19, system 1300 may advantageously usea transmission-type illumination to simplify its geometry. However,reflection-type illumination configurations as shown in FIGS. 13-18 mayalso be used depending on the application. In the illumination system,excitation light 1402 from light source 1310, such as a supercontinuumlaser source provided through an optic fiber, is collimated by lens 1330a, transmits through a first diffractive element 1600 a, and thenilluminates a sample placed on sample holder 1370. Diffractive element1600 a modulates the phase of excitation light 1402 transmitting throughit and structures excitation light 1402 for generating excitationpattern 1100. The phase modulation may render a plurality of wavelets ofthe transmitted excitation light 1430 with different directions and/orphases, generating a diffraction pattern in the far field. Thediffraction pattern focused on the sample is referred to as excitationpattern 1100.

FIG. 22 is a schematic representation of an exemplary diffractiveelement 1600 a. As shown in FIG. 22, in some embodiments, diffractiveelement 1600 a may be a 2-D array of diffractive lenses 1610. Forexcitation light 1402 having a single wavelength, diffractive element1600 a generates a 2-D array of excitation spots, one by eachdiffractive lens 1610. For excitation light 1402 having multiplediscrete wavelengths or a range of wavelengths, different wavelengths ofexcitation light 1402 are diffracted by each diffractive lens 1610 intoseveral beams travelling in different angular directions. Therefore,when focused on the sample, the different wavelengths of excitationlight 1402 may have focuses spatially shifted from one another in afirst lateral direction (e.g., vertical direction), thereby generatingexcitation pattern 1100 as shown in FIG. 11 or FIG. 12.

In some embodiments, diffractive lenses 1610 of diffractive element 1600a may be zone plates that have transparent and nontransparent bands,conventional gratings made by, e.g., binary lithography, grayscalelithography, or molding processes, or subwavelength gratings made bybinary lithography. In other embodiments, diffractive element 600 a maybe replaced with a 2-D lenslet array and a transmissive diffractiongrating that have the phase modulation capability for generatingexcitation pattern 1100 as described above.

In the detection system, fluorescent light 1408 emitted by the sample iscollected and collimated by objective 1360, transmits through dispersiveelement 1340 b, and is then focused onto imaging device 1380 by lens1330 b. Dispersive element 1340 b may spectrally disperse fluorescencelight 1408 in a second lateral direction (e.g., horizontal direction) asdescribed above. Dispersive element 1340 b may have the same featuresand functions as described above.

In some embodiments, system 1300 may include a second linear polarizer1390 c. Fluorescent light 1408 may pass through polarizer 1390 c. Whenexcitation light 1402 is linearly polarized, polarizer 1390 c may beused to substantially reflect the polarized excitation light and thusblock it from reaching imaging device 1380. In other embodiments, a setof notch filters or a single multi-notch filter (not shown) may be addedto the detection system along the optical axis.

Because diffractive element 1600 a does not have the digitalprogrammability as that of an SLM, either diffractive element 1600 a orsample holder 1370 may be translated in spatial dimensions to scanexcitation pattern 1100 across the field of view or the sample to obtaina complete 4-D hyperspectral-imaging dataset. The scanning scheme may bethe same as described above in reference to FIGS. 11 and 12. Differentareas in each scanning cell 1110 may be illuminated by spatiallyshifting excitation pattern 1100 in the vertical and horizontaldirections. At each spatial position of excitation pattern 1100, atleast one 2-D image 1200 of fluorescence emission spectra of theilluminated areas can be acquired. Then, a plurality of 2-D images 1200of fluorescence emission spectra can be acquired corresponding to aseries of excitation patterns 1100 laterally shifted from one anotherand used for reconstructing the 4-D hyperspectral-imaging dataset.

FIG. 20 is a schematic representation of another exemplary compactembodiment of system 1300. System 1300 as shown in FIG. 20 may allow foracquiring a 4-D hyperspectral-imaging dataset by performing scanning inone lateral direction. For example, system 1300 may include diffractiveelement 1600 a in the detection system and another diffractive element1600 b in the illumination system. As shown in FIG. 20, in theillumination system, excitation light 1402 from light source 1310 iscollimated by lens 1330 a, transmits through diffractive element 600 b,and then illuminates a sample placed on sample holder 1370.

FIG. 23 is a schematic representation of an exemplary diffractiveelement 1600 b. Diffractive element 1600 b may modulate the phase ofexcitation light 1402 transmitting through it and render wavelets oftransmitted excitation light 1430 having different directions and/orphases. In some embodiments, diffractive element 1600 b may include alinear array of diffractive cylindrical lenslets 1620. For excitationlight 1402 of a single wavelength, diffractive element 1600 b generatesa repeating pattern of single-colored stripes, one by each cylindricallenslet 1620. For excitation light 1402 having multiple discretewavelengths or a range of wavelengths, different wavelengths ofexcitation light 1402 are diffracted by each cylindrical lenslet 1620into several beams travelling in different angular directions.Therefore, when focused on the sample, different wavelengths ofexcitation light 1402 may have focuses spatially shifted from oneanother in a first lateral direction (e.g., vertical direction),generating a repeating pattern of a series of shifted different-coloredstripes. Depending on the spectrum of light source 1310, thedifferent-colored stripes may be connected or separated in the firstlateral direction. The repeating pattern of shifted different-coloredstripes are then illuminated on the sample.

In the detection system, rather than using dispersive element 1340 b,diffractive element 1600 a may be added and placed in front of imagingdevice 1380. Fluorescent light 1408 emitted by the sample is collectedand collimated by objective 1360, transmits through polarizer 1390 c,and is then imaged onto diffractive element 600 a by lens 1330 b.Diffractive lenses 1610 of diffractive element 1600 a may thenspectrally disperse the fluorescent light in a second lateral direction(e.g., horizontal direction) and image the spectrally dispersedfluorescent light 1410 to the 2-D sensor of imaging device 1380.

In some embodiments, the focal length of lens 1330 b is selected suchthat a diffraction-limited spot size of lens 1330 b at its focal planemay cover a plurality of pixels of the 2-D sensor of imaging device1380. This may affect the numerical aperture (NA), the focal ratio(f-ratio), and/or the magnification of lens 1330 b. For example, toincrease the diffraction-limited spot size of lens 1330 b, lens 1330 bmay have a longer focal length, a smaller NA or a larger f-ratio, and/ora greater magnification.

Diffractive element 1600 a may be designed or selected such that thediameters of its diffractive lenses 1610 are about the size of adiffraction-limited spot of lens 1330 b. Different wavelengths of thefluorescent light 1410 deflected and focused by each diffractive lens1610 may have focuses spatially shifted from one another in the secondlateral direction, generating an array of fluorescence emission spectraas shown in FIG. 11 or FIG. 12.

Embodiments of system 1300 as shown in FIG. 20 allows for acquiringfluorescence emission spectra in a 2-D image 1200 as shown in FIG. 11 orFIG. 12 for areas or locations on the sample illuminated by therepeating pattern of a series of laterally shifted different-coloredstripes. To acquire the excitation spectra, the repeating pattern may bescanned along the first lateral direction such that the areas orlocations on the sample previously illuminated by a colored stripe ofthe repeating pattern are illuminated by a different-colored stripe. Theshifting of the repeating pattern in the first lateral direction andsubsequent acquisition of a corresponding 2-D image 1200 may beperformed for a plurality times. In such instances, fluorescenceemission spectra corresponding to the excitation wavelengths for eacharea or location on the sample can be acquired and used forreconstructing the 4-D hyperspectral-imaging dataset.

In the embodiments of system 1300 as shown in FIG. 20, because therepeating pattern of a series of laterally shifted different-coloredstripes is continuous in the second lateral direction, the repeatingpattern may only need to be scanned along the first lateral direction toobtain the excitation spectra for all areas or locations within thefield of view. This may further improve the throughput and efficiency ofsystem 1300 for acquiring the 4-D hyperspectral-imaging dataset.

Along the second lateral direction, each area illuminated by thecontinuous colored stripes can be imaged to a diffractive lens 1610,which then disperses the fluorescent light and focuses it to imagingdevice 1380. In such instances, the spatial resolution along the secondlateral direction may depend on the size and focal length of diffractivelenses 1610, the focal lengths of lens 1330 b and objective 1360, and/orthe size of the 2-D sensor of imaging device 1380. In some embodiments,increasing the focal length of lens 1330 b may allow for using largerdiffractive lenses 1610. The spectral resolution along the secondlateral direction may depend on the width and/or focal length ofdiffractive lenses 1610, and the off-axis focal shifts generated bydiffractive lenses 1610 in the second lateral direction. For example,increasing groove density of diffractive lenses 1610 would increase thediffraction angles of the fluorescent light and thus the off-axis focalshifts, thereby increasing the spectral resolution in the second lateraldirection.

FIG. 21 is a schematic representation of another exemplary compactembodiment of system 1300 that provides the capability for measuringfluorescence polarization. As shown in FIG. 21, system 1300 may includetwo polarizers 1390 a and 1390 c. Polarizer 1390 a may be at a suitableplace along the optical axis in the illumination system, therebygenerating linearly polarized excitation light. Polarizer 1390 c may beat a suitable place along the optical axis in the detection system,thereby transmitting emitted fluorescent light 1408 having a givenvibration orientation. To perform fluorescence polarization assays, thetransmission axis of polarizer 1390 c may be rotated betweenorientations parallel and orthogonal to the vibration orientation of thelinearly polarized excitation light. 2-D images 1200 of the fluorescenceemission spectra of fluorescent light 1408 having vibration orientationsparallel to and orthogonal to that of the polarized excitation light maybe respectively acquired by imaging device 1380. The acquired 2-D images1200 may then be used for fluorescence polarization (or anisotropy)assay.

System 1300 as described herein may be utilized in a variety of methodsfor hyperspectral imaging. FIG. 24 is a flowchart of an exemplary method2400 for performing hyperspectral imaging or for acquiring ahyperspectral-imaging dataset of a sample. Method 2400 uses system 1300and features of the embodiments of system 1300 described above inreference to FIGS. 13-23.

At step 2402, light source 1310 having a discrete spectrum or acontinuous spectrum is provided and configured to emit excitation light1402 having one or more wavelengths. At step 2404, excitation light 1402is structured by SLM 1320 a to into a predetermined two-dimensionalpattern at a conjugate plane of a focal plane in the sample. At step2406, the structured excitation light, e.g., excitation light 1404reflected by SLM 1320 a, is spectrally dispersed by dispersive element1340 a in a first lateral direction. At step 2408, spectrally dispersedexcitation light 1406 is directed towards and focused on the sample,illuminating the sample in excitation pattern 1100 with the one or morewavelengths dispersed in the first lateral direction. At step 2410,fluorescent light 1408 collected from the sample is spectrally dispersedby dispersive element 1340 b in a second lateral direction. At step2412, spectrally dispersed fluorescent light 1410 is imaged to a 2-Dsensor of imaging device 1380.

Method 2400 may further include additional steps. For example, method2400 may include calibrating system 1300 before acquiring 2-D image1200. Various optical components in system 1300 may be suitablycalibrated and aligned such that focused 2-D images 1200 with reduced orminimum distortion can be acquired.

Method 2400 may further include polarizing excitation light 1402 to bedirected to the sample using a first polarizer, and substantiallyreflecting light collected from the sample having the same polarizationas that of the polarized excitation light using a second polarizer or apolarizing beamsplitter (PBS).

Method 2400 may further include illuminating the sample sequentially ina series of excitation patterns 1100 laterally shifted from one another,and obtaining a plurality of 2-D images 1200 of the spectrally dispersedemission light corresponding to the series of excitation patterns 1100,and reconstructing the plurality of 2-D images 1200 to provide a 4-Dhyperspectral-imaging dataset. As described above, each 2-D image 1200records an array of fluorescence emission spectra corresponding to eachlaterally shifted excitation pattern 1100.

Method 2400 may further include providing programmable artificialoptical pinholes at a plane conjugate to the focal plane by SLM 1320 b,forming a series of pinhole patterns by pixels of SLM 1320 b, andmatching the series of pinhole patterns to the series of excitationpatterns 1100. As described above, light collected from SLM 1320 b isimaged to imaging device 1380 using one or more lenses. A 2-D image 1200of the spectrally dispersed emission light may be acquired after eachlateral shift of excitation pattern 1100 and the formation of itsmatching pinhole pattern. Method 2400 may further include reconstructingthe 2-D images 1200 corresponding to the series of excitation patterns1100 to provide a 4-D hyperspectral-imaging dataset of the selectedfocal plane of the sample.

VIII. IMAGING USING SPATIALLY PHASE-SPECIFIED ILLUMINATION

It could be beneficial to control the spatial distribution of the phase,intensity, wavefront geometry, polarization, or other properties ofillumination applied to a sample in order to image the sample. Suchimaging of a sample (e.g., a biological sample) could be effected inorder to identify probes in the sample, to detect the location, color,or other properties of fluorophores in the sample (e.g., fluorophores ofsuch a probe), or to provide some other benefit. Controlling the spatialproperties of the applied illumination could include operating a spatiallight modulator to control the relative phase of the illumination acrossthe sample. Such a method for controlling the phase of illuminationapplied to a sample could be employed in addition to, or alternativelyto, other methods or apparatus for illumination of a sample describedelsewhere herein. Embodiments to produce such illumination may beimplemented using a microscope, such as a fluorescence microscope, aconfocal microscope, a transmission microscope, or a reflectancemicroscope, having one or more 2-D imaging devices, e.g., a CCD or CMOSsensor or camera. Alternatively, an optical imaging system may be builtaccording to embodiments of the present disclosure using suitableoptical elements.

Embodiments of the present disclosure allow for acquiring a 2-D image ofa focal plane in a sample using programmable artificial pinholes withadjustable size and spacing. A plurality of 2-D images can be acquiredat a plurality of focal planes and computationally reconstructed toobtain a 3-D or virtual volumetric image of a sample. Additionally,embodiments of the present disclosure allows for acquiring ahyperspectral confocal image dataset of a focal plane in the sample.

According to an aspect of the present disclosure, excitation lighthaving one or more wavelengths may be used to excite fluorophores in thesample. The excitation light may be emitted by a single-color lightsource or a multi-color light source. In some embodiments, thesingle-color light source may be a pulsed or a continuous“single-wavelength” laser that emits light with a very narrow spectrum.In other embodiments, the single-color light source may be the output ofa monochromator.

In some embodiments, the multi-color light source may have a continuousspectrum. For example, the multi-color light source may be a broadbandlight source, such as certain supercontinuum lasers, a white lightsource (e.g., a high-pressure mercury lamp, a xenon lamp, a halogenlamp, or a metal halide lamp), or one or more LEDs. In otherembodiments, the multi-color light source may have a discrete spectrum.For example, the multi-color light source may be a combination of pulsedor continuous “single-wavelength” lasers that emit light with verynarrow spectra.

According to an aspect of the present disclosure, excitation lightemitted by the light source may be structured for illuminating a subsetof areas on the sample in an excitation pattern using a first spatiallight modulator (SLM). To structure the excitation light, the first SLMmay modulate the phase or amplitude of the excitation light byselectively modulating, e.g., actuating or switching, its pixels. Thepixels could either be digital or analog in modulation. The first SLMmay be selected from a group of SLMs including a digital micromirrordevice (DMD), deformable mirrors (DM), a diffractive optical element, aliquid crystal device (LCD), and a liquid crystal-on-silicon (LCOS)device.

As described herein, an excitation pattern illuminated on a sample mayinclude a plurality of focused spots of excitation light (excitationspots). The excitation pattern may be an arbitrary pattern or apredetermined pattern, such as a 2-D array of excitation spotssimultaneously incident on the sample. Fluorophores or fluorescentmolecules in the sample illuminated by the excitation pattern may beexcited and subsequently emit fluorescent light.

In some embodiments, the excitation pattern may be scanned across thesample or the field of view by modulating the pixels of the first SLM.In other embodiments, an x-y translation stage may be used to scan theexcitation pattern across the sample or the field of view by moving thesample or an objective in lateral directions. The stage may be amotorized translation stage, a piezoelectric translation stage, or anysuitable stage that allows for lateral linear movement.

According to an aspect of the present disclosure, systems and methodsaccording to the present disclosure allow for confocal opticalsectioning. This allows for acquisition of images for a plurality offocal planes along an axial direction of the sample. In someembodiments, an image of a desired focal plane may be acquired byimplementing one or more optical pinholes at a plane conjugate to thefocal plane. The optical pinholes may be programmable artificialpinholes formed by pixels of a second SLM. The second SLM may beselected from a group of SLMs including a digital micromirror device(DMD), a liquid crystal device (LCD), and a liquid crystal-on-silicon(LCOS) device.

In some embodiments, a pinhole pattern may be formed by the pixels ofthe second SLM by selectively modulating or switching its pixels tomatch the excitation pattern of the excitation light. Advantageously,the pinhole pattern may allow for confocal imaging of a plurality ofareas on the sample simultaneously illuminated by the excitationpattern. This may increase the speed and/or throughput of acquiringconfocal images across the sample at the focal plane comparing totraditional confocal microscopes that use sequential point-by-pointscanning. Additionally, a degree of optical sectioning or confocalitymay be advantageously adjustable as needed by changing the size and/orseparation of the artificial pinholes formed by the second SLM, allowingfor adjusting the degree of depth selectivity of the desired focalplane.

As described herein, fluorophores are used in this disclosure as anexemplary optical label in a sample. Descriptions in references tofluorophores are equally applicable to other types of optical labelsconsistent with the embodiments of this disclosure. For example, theexcitation light emitted from the light source may also excite othertypes of optical labels, which upon excitation, may emit light with anemission spectrum. Therefore, fluorescent light and fluorescenceemission spectrum used in the descriptions in this disclosure may alsobe used to represent the emission light and emission spectra of otheroptical labels.

According to an aspect of the present disclosure, systems and methodsaccording to the present disclosure allows for hyperspectral imaging.Fluorescent light emitted by the fluorophores excited by the excitationlight in a given area of the sample may be spectrally dispersed in agiven lateral direction (e.g., the horizontal direction or the verticaldirection). At least one dispersive element may be employed tospectrally disperse the fluorescent light into a fluorescence emissionspectrum corresponding to that given area. The fluorescence emissionspectra of a subset of areas on the sample may be acquired as a 2-Dimage in one exposure by the 2-D imaging device.

In some embodiments, fluorescence emission spectra of all the areasacross the sample or across a field of view may be acquired by scanningthe excitation pattern across the sample or the field of view. At eachspatial location of the excitation pattern, a 2-D image of thefluorescence emission spectra corresponding to the excitation patternmay be acquired (e.g., each fluorescence emission spectrum correspondingto an excitation spot of the excitation pattern). Advantageously, ahyperspectral image dataset of the sample may be computationallyreconstructed from a plurality of such 2-D images of the fluorescenceemission spectra. Additionally, by forming a pinhole pattern thatmatches the excitation pattern using the second SLM during the scanning,a hyperspectral confocal image dataset of the sample can be obtained.

Reference will now be made in detail to embodiments and aspects of thepresent disclosure, examples of which are illustrated in theaccompanying drawings. Where possible, the same reference numbers willbe used throughout the drawings to refer to the same or like parts.

As described herein, to illustrate different wavelengths or frequenciesof light, different densities of dotted texture are used in the attacheddrawings. Higher densities correspond to longer wavelengths or lowerfrequencies of light. Additionally, vertical and/or horizontaldirections are used as examples for illustrating lateral or transversaldirections.

FIG. 25 is a schematic representation of an exemplary confocal imagingsystem 2500. In some embodiments, system 2500 may be a fluorescencemicroscope, a transmission microscope, a reflectance microscope, or aconfocal fluorescence microscope. Embodiments of the present disclosureare applicable to other suitable microscopy techniques for performingconfocal imaging and/or hyperspectral imaging.

As shown in FIG. 25, system 2500 may include an illumination system anda detection system. The illumination system may include a light source2510, a first SLM 2540, and one or more lenses, e.g., lenses 2520 a,2520 b, and 2520 c. The illumination system may further include ahalf-wave plate 2530, an optical beam dump 2525, and/or an opticalfilter 2600 a. The detection system may include a second SLM 2550, a 2-Dimaging device 2590, and one or more lenses, e.g., lens 2520 d, 2520 e,and 2520 f. The detection system may further include a dispersiveelement 2610 and/or an optical filter 2600 b. Depending on its layout,geometry, and/or application, system 2500 may further include abeamsplitter 2560, an objective 2570, and a sample holder 2580 where asample to be imaged is placed. System 2500 may include other opticalelements, such as mirrors, beam dumps, an x-y translation stage, az-axis translation stage or a tunable liquid lens (not shown), etc.

As described herein, an optical axis of system 2500 may define a pathalong which the excitation light and emitted fluorescent light from thesample propagate through system 2500.

In the illumination system, as shown in FIG. 25, light source 2510 emitsexcitation light 2501, which is directed to SLM 2540. Excitation light2501 may be collimated and/or expanded using one or two lenses, e.g.,lens 2520 a or a pair of lenses 2520 a. SLM 2540 may structurecollimated excitation light 2502 through modulating the phase oramplitude of excitation light 2502 by selectively actuating or switchingits pixels. SLM 2540 may be a transmission type or a reflection typeSLM. While a reflection type SLM 2540 is used in the exemplaryembodiment shown in FIG. 25, a transmission type SLM 2540 mayalternatively be used consistent with the present disclosure. Thegeometry of the illumination system may be suitably designed based onthe type of SLM 2540.

As shown in FIG. 25, when SLM 2540 is a reflection type SLM, at least aportion of the pixels of SLM 2540 reflect excitation light 2502 anddirect the reflected excitation light 2503 along the optical axis ofsystem 2500. In some embodiments, excitation light 2503 may be directedby SLM 2540 straight towards beamsplitter 2560 and/or objective 2570. Inother embodiments, as shown in FIG. 25, reflected excitation light 2503may pass through one or more relay lenses, e.g., lenses 2520 b and 2520c, before reaching beamsplitter 2560 and/or objective 2570. Objective2570 then focuses the excitation light to a sample placed on sampleholder 2580.

In the detection system, as shown in FIG. 25, fluorescent light 2504emitted by excited fluorophores in the sample is collected and/orcollimated by objective 2570. Fluorescent light 2504 may pass throughbeamsplitter 2560 and lens 2520 d along the optical axis of system 2500.SLM 2550 may be placed at about a plane conjugate to a focal planelocated at a desired depth in the sample along the optical axis. Forexample, objective 2570 and lens 2520 d may form an imagingconfiguration. When SLM 2550 is a reflection type SLM, SLM 2550 mayreflect at least a portion of fluorescent light 2504 and direct thereflected fluorescent light 2505 along the optical axis of system 2500towards imaging device 2590. Reflected fluorescent light 2505 may pass apair of tube lenses, e.g., lenses 2520 e and 2520 f, before reaching a2-D sensor of imaging device 2590.

As described herein, while a reflection type SLM 2550 is used in theexemplary embodiment shown in FIG. 25, a transmission type SLMs 2550 mayalternatively be used consistent with the present disclosure. Thegeometry of the detection system may be suitably designed based on thetype of SLM 2550.

Functions and the working principles of various components of system2500 are described in detail below.

Light Source

As described above, light source 2510 may be a single-color light sourceor multi-color light source. In some embodiments, excitation light 2501emitted by light source 2510 may be linearly polarized. Additionally oralternatively, excitation light 2501 may be collimated by lens 2520 aand become collimated excitation light 2502 before being incident on SLM2540.

In some embodiments, collimated excitation light 2502 may pass throughhalf-wave plate 2530. Half-wave plate 2530 may change the polarizationdirection of a linearly polarized excitation light. For example, whenSLM 2540 is a LCD or LCOS device, half-wave plate 2530 may rotate thepolarization direction of the linearly polarized excitation light to bealigned in parallel with the orientation of the liquid crystal moleculesin SLM 2540. This may increase the efficiency of reflection and/ormodulation of the excitation light by the pixels of SLM 2540.

In some embodiments, light source 2510 may be operably connected to acontroller (not shown) having a processor and a computer-readable mediumthat stores instructions or operational steps. These instructions orsteps, when executed by the processor, may modulate the operationalstates of light source 2510. For example, the processor may activate ordeactivate light source 2510, modulate the duration of a pulse of apulsed light source 2510, and/or switch or tune the emission wavelengthsof light source 2510.

Spatial Light Modulator for Structuring Excitation Light

As described above, to structure excitation light 2502 for illuminatingthe sample in an excitation pattern, SLM 2540 may modulate the amplitudeor phase of excitation light 2502 by selectively modulating its pixelsbetween operational states.

Amplitude Modulation

In some embodiments, the amplitude of excitation light 2502 may bemodulated by SLM 2540. For example, SLM 2540 may be a reflection typeLCD or LCOS device. The LCD or LCOS device may be placed at a conjugateplane to the sample. In such instances, only one of lenses 2520 b and2520 c may be placed between SLM 2540 and objective 2570. For example,lens 2520 b may be used as a tube lens and combined with objective 2570to form an imaging configuration. SLM 2540 may be placed at about onefocal length before lens 2520 b.

Pixels of SLM 2540 may create an amplitude modulation pattern bymanipulating the polarization of excitation light incident on thepixels. The amplitude modulation pattern may be imaged onto the sampleas an excitation pattern by lens 2520 b and objective 2570, for example.Depending on the focal lengths of lens 2520 b and objective 2570, theexcitation pattern may be a magnified or de-magnified image of theamplitude modulation pattern.

To create the amplitude modulation pattern, pixels of SLM 2540 may beelectrically modulated between an “on” state and an “off” state in apixel-by-pixel fashion. The “on” pixels may rotate the polarizationdirection of linearly polarized light by about 90° while the “off”pixels do not perform the rotation. In such instances, a first linearpolarizer (not shown) may be used to linearly polarize excitation light2502. A second linear polarizer or a polarizing beamsplitter (PBS) (notshown) may be used to pass excitation light 2503 reflected by the “on”pixels and block excitation light 2502 reflected by the “off” pixels.

A disadvantage of modulating the amplitude of excitation light 2502using SLM 2540 is the loss of light during the modulation. This isbecause most of the pixels of SLM 2540 are typically in the “off” state.Accordingly, most of excitation light 2502 is steered away from theoptical axis and would not reach the sample, and thus is lost.

Phase Modulation

To increase the efficiency of utilizing excitation light 2502, SLM 2540may modulate the phase of excitation light 2502 to generate theexcitation pattern. In such instances, both lenses 2520 b and 2520 c maybe placed between SLM 2540 and objective 2570. SLM 2540 may be areflection type LCD or LCOS device, for example. The LCD or LCOS devicemay be placed at an aperture plane, which may be a conjugate plane tothe back aperture of objective 2570 or a Fourier plane to the sample.For example, lenses 2520 b and 2520 c may form an imaging configuration.Lens 2520 b may be located about one focal length behind SLM 2540. Lens2520 c may be located about two focal lengths behind lens 2520 b.Objective 2570 may be located about one focal length behind lens 2520 c.

The pixels of SLM 2540 may form a custom phase modulation pattern tomodulate the wavefront of excitation light 2502. Upon the reflection ofexcitation light 2502 by SLM 2540, phases at different locations of thewavefront of the reflected excitation light 2503 may be selectivelychanged according to the phase modulation pattern. In some embodiments,pixels of SLM 2540 may be electrically modulated between an “on” stateand an “off” state in a pixel-by-pixel fashion. If pixels of SLM 2540are in the “on” state, they may change the phase of the reflected lightby changing the optical path length of light traveled in the liquidcrystal; and if they are in the “off” state, they may not change thephase of the reflected light. This allows the phase modulation patternformed by the pixels of SLM 2540 to be digitally customized as needed.In other embodiments, pixels of SLM 2540 may have multiple states orlevels of phase adjustment (e.g., 256 levels between 0 and 2π) and maybe individually modulated to desired states or levels. Advantageously,increasing the states or levels of adjustment of the pixels increasesthe continuity of the adjustment of the phase modulation pattern andthus the adjustment of the phase of excitation light 2503, and mayfurther reduce undesirable diffraction orders in the excitation pattern.

The phase modulation may render wavelets of reflected excitation light2503 having different directions and/or phases. As reflected excitationlight 2503 propagates along the optical axis, each of the lenses 2520 band 2520 c and objective 2570 may perform Fourier Transform on thewavefront of reflected excitation light 2503. A diffraction pattern maythen be formed at the focal plane of objective 2570. This diffractionpattern is referred to herein as the excitation pattern when illuminatedon the sample.

In some embodiments, optical beam dump 2525 may be placed along theoptical axis between lenses 2520 b and 2520 c, e.g., about a focallength behind lens 2520 b or at a conjugate plane of the sample. Thismay allow the lower-order diffraction spots, e.g., zero-order and/orfirst-order diffraction spots, of a diffraction pattern formed byreflected excitation light 2503 at the location of optical beam dump2525 to be substantially absorbed and blocked from reaching the sample.Because the excitation pattern is an image of the diffraction patternformed at the location of optical beam dump 2525, the intensity oflower-order diffraction spots of the excitation pattern illuminated onthe sample would be substantially reduced. Since the lower-orderdiffraction spots, e.g., zero-order and/or first-order diffractionspots, are typically brighter than other orders of diffraction spots,the use of optical beam dump 2525 may advantageously improve theuniformity of the intensity of the excitation pattern across the fieldof view.

As described above, the phase modulation pattern is at or approximatelyat a Fourier plane to the sample. In such instances, the electricalfield of reflected excitation light 2503, whose phase has been modulatedby the phase modulation pattern of SLM 2540, is further subject toFourier Transforms by the lenses 2520 b and 2520 c and objective 2570before it illuminates the sample in a desired excitation pattern. Insome embodiments, the excitation pattern may be an intensity profile ofthe wavefront of the transformed excitation light with a desired phaseprofile. The desired phase profile may be predetermined to increase thediffraction efficiency of the excitation light.

In some embodiments, computer algorithms, e.g., the Gerchberg-Saxton(GS) algorithm, may be used to generate the phase modulation patternthat would result in a desired excitation pattern. Further, customizedcomputer algorithms may be used to generate time-varying phasemodulation patterns for scanning or translating the desired excitationpattern across the field of view.

Advantageously, modulating the phase of excitation light 2502 wouldallow it to propagate with substantially uniform intensity in the nearfield of SLM 2540 and thus reduce loss of excitation light 2502. Themodulated excitation light may then form a customizable or programmableexcitation pattern when illuminated on the sample in the far field.Therefore, comparing to modulating the amplitude of excitation light2502 as described above, modulating the phase of excitation light 2502to create a desired excitation pattern may substantially increase theefficiency of illumination of system 2500 by reducing loss of excitationlight 2502.

SLM 2540 may alternatively be a transmission type device implementedalong the optical axis. The geometry of the illumination system may besuitably designed such that the amplitude or phase modulation patternformed by the pixels of the device may modulate the amplitude or phaseof excitation light 2502 similarly as described above.

Whether SLM 2540 modulate the amplitude or phase of excitation light2502, the excitation pattern illuminated on the sample can be programmedand customized as needed by modulating pixels of SLM 2540 between twooperational states in a pixel-by-pixel fashion. Further, the excitationpattern may be translated or shifted across the sample or a field ofview in a given spatial direction, such as the horizontal or verticaldirection, by scanning or changing the modulation of the pixels of SLM2540. For example, when SLM 2540 is located at a Fourier plane of thesample for modulating the phase of excitation light 2502, the excitationpattern may be scanned by changing the slope of a linear phase rampalong a spatial direction. This advantageously allows for scanning theexcitation pattern across the field of view of system 2500 withoutmoving the sample and/or sample holder 2580 using an x-y translationstage.

In some embodiments, depending on the type and modulation features ofthe pixels of SLM 2540, excitation light 2502 may be directed towardsSLM 2540 at a predetermined angle relative to a plane of SLM 2540. Thepredetermined angle may depend on the type of SLM 2540 and/or thegeometry of system 2500. For example, when SLM 2540 is a reflection typeSLM, excitation light 2502 may be directed towards SLM 2540 at an anglesuch that reflected excitation light 2503 propagates along the opticalaxis of system 2500.

In some embodiments, SLM 2540 may be operably connected to a controller(not shown) having a processor and a computer-readable medium thatstores instructions or operational steps. These instructions or steps,when executed by the processor, may modulate the operational states ofthe pixels of SLM 2540 to form a desired excitation pattern and/or totranslate the excitation pattern in a desired spatial direction over apredetermined distance across the field of view.

Confocal Optical Sectioning

As described above, system 2500 allows for confocal optical sectioning,which allows for selecting the depth of a focal plane in the sample. Thedepth of the focal plane may be selected by introducing one or moreoptical pinholes at a plane conjugate to the selected focal plane.

SLM 2550 is used for achieving confocal optical sectioning. As describedabove, SLM 2550 may be placed at about a plane conjugate to a focalplane located at a desired depth in the sample along the optical axis.Lens 2520 d may be used as a tube lens and together with objective 2570may form an imaging configuration. For example, as shown in FIG. 25,lens 2520 d may be located behind objective 2570 and SLM 2550 may belocated about one focal length behind lens 2520 d. The space between theback aperture of objective 2570 and lens 2520 d is a collimated space,which may be adjusted as need based on various factors, such as thegeometry of system 2500 and a desired location of a minimum beamaperture. In some embodiments, lens 2520 d is placed about one focallength behind objective 2570.

In some embodiments, SLM 2550 may be a digital micromirror device (DMD)having an array of multiple micromirrors 2552. These micromirrors may beindividually actuated to switch between two operational positions, an“on” position and an “off” position. When a micromirror is configured tobe in the “on” position, fluorescent light 2504 from the focal plane inthe sample is reflected to propagate along the optical axis as reflectedfluorescent light 2505, which is directed to imaging device 2590. When amicromirror is configured to be in the “off” position, fluorescent light2504 is reflected towards a direction deviated from the optical axis andis not directed to imaging device 2590. In some embodiments, fluorescentlight 2504 reflected by the “off” micromirrors may be directed to otheroptical elements, such as a mirror or a beam dump (not shown).

In some embodiments, the micromirrors are of a square shape having alength of its sides ranging from about a few micrometers to about 10 μm.Other shapes and sizes of the micromirrors are also possible and may besuitably used. The DMD is typically capable of changing or alternatingthe “on” and “off” positions of the micromirrors very rapidly.

In some embodiments, a single micromirror of the DMD may be referred toas a single pixel. In other embodiments, a plurality of micromirrors maybe referred to as a single pixel. For example, a group of immediatelyadjacent micromirrors may be referred as a single pixel and may bemodulated or actuated in unison.

Pixels of SLM 2550 may be selectively actuated or switched to “on” or“off” positions to form a pinhole pattern matching (conjugating) theexcitation pattern illuminated on the sample. The pinhole pattern mayinclude a plurality of artificial optical pinholes at the conjugateplane and reject out-of-focus fluorescent light from the sample.Therefore, out-of-focus fluorescent light would not pass through thedetection system and are substantially removed or eliminated from theacquired image by imaging device 2590.

The size and separations of the artificial pinholes in the pinholepattern are programmable, and may be customized based on the excitationpattern and the magnification of the imaging configuration formed byobjective 2570 and lens 2520 d. For example, an artificial pinhole inthe pinhole pattern may be formed by an array of “on” pixels to matchthe size of an excitation spot in the excitation pattern.

The fluorescent light 2505 reflected by the “on” pixels of SLM 2550 maythen be imaged to imaging device 2590 by lenses 2520 e and 2520 f. Forexample, lens 2520 e may be located about one focal length beyond theimage produced by lens 2520 d (e.g., about one focal length behind SLM2550) such that it re-collimates reflected fluorescent light 2505.Imaging device 2590 may be located about one focal length behind lens2520 f or at a conjugate plane of SLM 2550. Because the fluorescentlight is collimated in the space between lenses 2520 e and 2520 f, thedistance between lenses 2520 e and 2520 f may be adjusted as desired. Insome embodiments, lens 2520 f may be about two focal lengths behind lens2520 e such that a plane midway between lenses 2520 e and 2520 f isconjugate to an exit pupil of system 2500.

By digitally changing and/or laterally shifting the excitation patternusing SLM 2540 and the matching pinhole pattern correspondingly usingSLM 2550, the whole field of view may be scanned for acquiring aconfocal image. By further scanning the field of view across the sample,the whole sample can be scanned to obtain a complete confocal imagedataset of the sample.

In some embodiments, imaging device 2590 may be suitably tilted toreduce aberrations and thus improve the quality of the acquired images.This is at least because the “on” pixels of SLM 2550 may directreflected fluorescent light 2505 at an angle that is not perpendicularto the surface plane of SLM 2550 such that an image plane formed bylenses 2520 e and 2520 f may be tilted. Aberrations caused by thistilting effect may be compensated by properly tilting imaging device2590.

To change or select a depth of the focal plane, in some embodiments,sample holder 2580 may be installed on the z-axis translation stage. Thedesired depth of the focal plane may be selected by moving sample holder2580 along the optical axis using the z-axis translation stage.Alternatively, objective 2570 may be installed on the z-axis translationstage and the desired depth of the focal plane may be selected by movingobjective 2570 along the optical axis. As describe herein, the z-axistranslation stage may also include x-y translation capability to movethe field of view of system 2500 across the sample in lateraldirections.

In some embodiments, when SLM 2540 is at a Fourier plane for modulatingthe phase of excitation light 2502, the focal depth may be adjusted bychanging the phase modulation pattern formed by the pixels of SLM 2540.In such instances, excitation light 2503 modulated by the pixels of SLM2540 may, upon reflection, include a superposition of slightly divergingor converging beams determined by the phase modulation pattern.Depending on their degree of divergence or convergence, these beamswould focus at increased or reduced depth after passing through themicroscope objective.

In other embodiments, the desired depth of the focal plane may beselected by tuning the focus of a tunable liquid lens (not shown) placedbehind objective 2570. As described herein, the z-translation stage, thetunable liquid lens, and/or the phase modulation pattern of SLM 2540 maybe controlled by a computer program to achieve autofocusing.

Advantageously, a degree of confocality may be adjusted as needed bychanging the size and/or separation of the artificial pinholes formed bySLM 2550. For example, increasing the sizes of the pinholes byincreasing the number of pixels in the pinholes and/or reducing thepinhole spacing may reduce the degree of confocality and thus the degreeof depth selectivity of the desired focal plane. On the other hand,decreasing the size of the pinholes by reducing the number of pixels inthe pinholes and/or increasing the pinhole spacing may increase thedegree of confocality and thus the degree of depth selectivity of thedesired focal plane. In some embodiments, the depth selectivity may beproportional to the ratio of the number of “off” and “on” pixels of SLM2550. Therefore, SLM 2550 may advantageously allow for switching betweenwide-field imaging and confocal imaging as desired by convenientlyadjusting the pinhole size and/or separation.

Additionally, the pinhole pattern formed by pixels of SLM 2550advantageously allows for confocal imaging of a plurality of areas onthe sample simultaneously illuminated by the excitation patterngenerated by SLM 2540. This may increase the speed and/or throughput ofacquiring a confocal image dataset across the sample at the desiredfocal plane comparing to traditional confocal microscopes that usesequential point-by-point scanning.

Hyperspectral Imaging Capability

In some embodiments, hyperspectral imaging capability may beadvantageously added to system 2500 to allow for acquiring ahyperspectral-imaging dataset at a selected focal plane in the sample. Ahyperspectral-imaging dataset may be represented in three-dimensions(3-D): two spatial directions (horizontal direction and verticaldirection) and one spectral dimension (λ). Information in the spectraldimension of a hyperspectral-imaging dataset may reflect fluorescenceintensities as a function of a range of emission wavelengths of thefluorophores in the sample.

Hyperspectral imaging capability may be achieved by using dispersiveelement 2610 in system 2500. For example, dispersive element 2610 may belocated in the collimated space between lenses 2520 e and 2520 f.Dispersive element 2610 may be a diffraction grating or a prism, such asa non-deviating prism (e.g., Amici prisms or double Amici prisms).Dispersive element 2610 may spectrally disperse fluorescent light 2505reflected from SLM 2550 along a given lateral direction. Spectrallydispersed fluorescent light 2506 then passes through lens 2520 f and isacquired by imaging device 2590.

FIG. 26 is a graphical illustration for an exemplary scheme forperforming hyperspectral confocal imaging, according to embodiments ofthe present disclosure. In some embodiments, when system 2500 is in amonochromatic imaging mode, imaging device 2590 may acquire an image offluorescent light 2505 reflected by a pinhole pattern formed on SLM2550. The pinhole pattern conjugates an excitation pattern illuminatedon the sample. For example, a 2-D image 2592 acquired by imaging device2590 may show a 2-D array 2620 of fluorescent spots 2622 correspondingto a 2-D array of excitation spots in the excitation pattern.

In other embodiments, when system 2500 is in a hyperspectral imagingmode, fluorescent light 2505 reflected by SLM 2550 is spectrallydispersed by dispersive element 2610 in a given direction, e.g., thehorizontal direction. In such instances, 2-D image 2592 acquired byimaging device 2590 may show a 2-D array 2630 of fluorescence emissionspectra 2632. Each fluorescence emission spectrum 2632 may be dispersedin the horizontal direction and correspond to an excitation spot of theexcitation pattern at a different spatial location on the sample.

As described above, the excitation pattern may be laterally shifted,e.g., in the vertical and horizontal directions to scan across the fieldof view or the sample. At each spatial position of the excitationpattern, array 2630 of fluorescence emission spectra 2632 correspondingto the areas on the sample illuminated by the excitation pattern can beacquired in a 2-D image 2592. A plurality of 2-D images 2592 offluorescence emission spectra may be acquired corresponding to a seriesof excitation patterns laterally shifted from one another and thencomputationally reconstructed to obtain a hyperspectral-imaging dataset.

Therefore, by digitally changing and/or laterally shifting theexcitation pattern and the matching pinhole pattern on SLM 2550correspondingly, the whole field of view may be scanned for acquiring ahyperspectral-imaging dataset of a sample at a focal plane. By furtherscanning the field of view across the sample, the whole sample can bescanned to obtain a complete hyperspectral-imaging dataset of the sampleat the focal plane.

The spatial separation, horizontal and/or vertical, between excitationspots of an excitation pattern may be predetermined based on variousfactors, such as the excitation wavelengths, the size of the sample, thefield of view of system 2500, the desired measurement throughput,spatial resolution, and/or speed, and the amounts of spectral dispersionof fluorescent light 2506. For example, the spatial separation betweenthe excitation spots in the horizontal direction may be predeterminedbased on the range of fluorescence emission spectra 2632 in thehorizontal direction such that the fluorescence emission spectra 2632 donot overlap with each other in the horizontal direction in 2-D image2592.

The degree of spectral dispersion caused by dispersive element 2610 maybe predetermined based on various factors, such as the spectral range offluorescent light 2505, the size of the sample or the field of view, thesize of imaging device 2590, the desired spectral resolution, and theapplication of system 2500.

In some embodiments, the degree of spectral dispersion caused bydispersive element 2610 may be advantageously adjustable. For example,dispersive element 2610 may be a pair of double Amici prisms placedalong the optical axis of system 2500. At least one of the pair ofdouble Amici prisms is rotatable relative to the other around theoptical axis. The rotation of the double Amici prisms relative to eachother may allow for continuous control of the amount and/or angularorientation (e.g., dispersion angles) of the spectral dispersion offluorescent light 2506.

Lenses and Objective

Various lenses of system 2500, such as lenses 2520 a-2520 f, may beachromatic, such as achromatic doublets or triplets, to limit or reducethe effects of chromatic and/or spherical aberration of the system.Further, objective 2570 of system 2500 may be achromatic. Alternativelyor additionally, objective 2570 may be an infinity-corrected objectivesuch that objective 2570 may form a desired focus (e.g., focused spotsor focused pattern) of a collimated light beam entering from its backaperture. Using achromatic lenses and/or achromatic orinfinity-corrected objective may allow fluorescent light of differentwavelengths from a focal plane in the sample to similarly form a focusedimage at imaging device 2590. Therefore, using achromatic lenses and/orachromatic objective may improve the quality of confocal images acquiredby system 2500.

Optical Filters and Beamsplitter

In some embodiments, optical filter 2600 a may be added in theillumination system along the optical axis. Optical filter 2600 a may bea clean-up filter that substantially transmits desired wavelengths ofexcitation light 2502 and blocks unwanted wavelengths. For example,optical filter 2600 a may have a narrow passband ranging for about a fewnanometers to block noisy spontaneous emission from light source 2510 orto substantially reduce background noise.

Because the intensity of excitation light 2502 may be orders ofmagnitude stronger than fluorescent light 2504, excitation light 2502reflected and/or scattered by the sample and/or sample holder 2580 mayenter the detection system and affect the detection or acquisition ofthe fluorescent light by imaging device 2590. Therefore, embodiments ofthe present disclosure may reduce or block excitation light 2502 frompropagating into the detection system as described below.

In some embodiments, beamsplitter 2560 may be used to block excitationlight 2502 from propagating towards imaging device 2590. Beamsplitter2560 may be a long-pass dichroic beamsplitter that substantiallyreflects the wavelengths of excitation light 2502 and transmits at leasta portion of the wavelengths of fluorescent light 2504. The spectrum ofexcitation light 2502 typically ranges from the ultraviolet through thevisible spectra, and the spectrum of fluorescent light 2504 typicallyranges from the visible into the near infrared spectra. Therefore, thelong-pass dichroic beamsplitter may block wavelengths of excitationlight 2502 and transmit a range of wavelengths of fluorescent light2504.

Alternatively or additionally, optical filter 2600 b may be added in thedetection system along the optical axis. Optical filter 2600 b may be anotch filter that may substantially reflect the wavelengths or a narrowspectral band of excitation light 2502, thereby blocking excitationlight 2502 from reaching imaging device 2590.

In other embodiments, when excitation light 2502 is linearly polarized,beamsplitter 2560 may be a polarizing beamsplitter (PBS). The PBS may beselected such that it reflects light having a polarization directionsame as that of the linearly polarized excitation light and to transmitlight having a polarization direction perpendicular to that of thepolarized excitation light. Most of the excitation light collected byobjective 2570 would therefore reflect from this PBS and would not reachimaging device 2590. In some instances, both the sample and objective2570 may depolarize reflected or scattered excitation light to a smalldegree, and thus undesirably allow some excitation light to transmitthrough the PBS and enter the detection system.

Imaging Device

Imaging device 2590 may include a suitable 2-D sensor located at animage plane conjugate to a selected focal plane in the sample. Thesensor could be implemented with a CMOS sensor, a CCD sensor, a 2-Darray of silicon avalanche photodiodes (APDs), or other suitable typesof 2-D sensors.

Imaging device 2590 may be operatively connected to a controller or acomputing device (not shown) that controls its operation. For example,the controller (not shown) may have a processor and one or morecomputer-readable media that stores instructions or operational steps.The instructions or operational steps, when executed by the processor,may operate the exposure of imaging device 2590, acquire 2-D images2592, and/or store the datasets of 2-D image 2592 to a memory. Thecomputer-readable medium may further store instructions or operationalsteps that, when executed by the processor, may perform data processingof the acquired 2-D image datasets and/or reconstruct a confocal imageand/or a hyperspectral-imaging dataset from the 2-D image datasets.

System 2500 as described herein may be utilized in a variety of methodsfor confocal and/or hyperspectral imaging. FIG. 27 is a flowchart of anexemplary method 2700 for performing confocal imaging or for acquiring aconfocal image of a sample. Method 2700 uses system 2500 and features ofthe embodiments of system 2500 described above in reference to FIGS. 25and 26.

At step 2702, light source 2510 is provided and configured to emitexcitation light 2501 having one or more wavelengths. At step 2704,excitation light 2501 is collimated by lens 2520 a and become collimatedexcitation light 2502. At step 2706, collimated excitation light 2502 isstructured or modulated by being applied with a predetermined phasemodulation pattern formed by pixels of SLM 2540. At step 2708, thestructured excitation light is directed towards the sample andilluminates the sample in a two-dimensional excitation pattern. Theexcitation pattern is located at a Fourier plane of the phase modulationpattern. At step 2710, emission light collected from a focal plane inthe sample is imaged to imaging device 2590. The focal plane can beconjugate to or at a conjugate plane of a pinhole pattern formed by thepixels of SLM 2550.

Method 2700 may further include additional steps. For example, method2700 may include calibrating system 2500 before acquiring 2-D image2592. Various optical components in system 2500 may be suitablycalibrated and aligned such that focused 2-D images 2592 with reduced orminimum aberration and/or distortion can be acquired.

Method 2700 may further include spectrally dispersing fluorescent light2504 collected from the sample in a lateral direction using dispersiveelement 2610. Spectrally dispersed fluorescent light 2506 may beacquired in a 2-D image 2592 by imaging device 2590.

Method 2700 may further include illuminating the sample sequentially ina series of excitation patterns laterally shifted from one another andforming a series of pinhole patterns matching the series of excitationpatterns.

In some embodiments, method 2700 may further include obtaining aplurality of 2-D images 2592 of the emission light 2505 corresponding tothe series of excitation patterns, and reconstructing the plurality of2-D images 2592 to provide a confocal image. As described above, a 2-Dimage 2592 may be acquired after each lateral shift of excitationpattern and the formation of its matching pinhole pattern. Each 2-Dimage 2592 may record an array 2620 of fluorescent spots 2622corresponding to each laterally shifted excitation pattern.

In other embodiments, method 2700 may further include obtaining aplurality of 2-D images 2592 of the spectrally dispersed emission light2506 corresponding to the series of excitation patterns, andreconstructing the plurality of 2-D images 2592 to provide ahyperspectral confocal image dataset. As described above, a 2-D image2592 of the spectrally dispersed emission light may be acquired aftereach lateral shift of excitation pattern and the formation of itsmatching pinhole pattern. Each 2-D image 2592 may record an array 2630of fluorescent emission spectra 2632 corresponding to each laterallyshifted excitation pattern.

IX. EXAMPLE CONTROLLABLE OPTICALLY DISPERSIVE ELEMENT

Various embodiments herein describe the use of chromatically dispersiveelements or systems (e.g., prisms, diffraction gratings, SLMs) todisperse light (e.g., illumination light used to illuminate a sample,image light received from a sample) according to wavelength. Thisdispersion can be employed to facilitate imaging of a sample (e.g., abiological sample) in order to identify probes in the sample, to detectthe location, color, or other properties of fluorophores in the sample(e.g., fluorophores of such a probe), or to provide some other benefit.In some examples (e.g., in the systems described above), it could beadvantageous to apply such dispersion using “direct vision” or“non-deviating” dispersive elements to achieve flexible adjustment ofthe magnitude and/or orientation of the applied dispersion. Embodimentsof the present disclosure may be implemented in a spectrometer, e.g., animaging spectrometer, a monochromator, a spectral analyzer, amicroscope, e.g., a fluorescence microscope, a confocal microscope, atransmission microscope, a reflectance microscope, etc., or a spectralimaging system, e.g., a hyperspectral imaging system. Alternatively,embodiments of the present disclosure may be implemented in a customizedoptical system built using suitable optical elements.

According to an aspect of the present disclosure, an optical system isprovided for dispersing an optical beam having one or more wavelengths.The optical system may include a pair of non-deviating dispersiveelements aligned along an optical axis. In some embodiments, the opticalsystem may collimate the input optical beam before the dispersion.

In some embodiments, the optical beam may be an excitation light beamfor illuminating a sample or an emission light beam collected from asample. Additionally or alternatively, the optical beam may be filteredto have a desired spectrum before entering the optical system.

According to an aspect of the present disclosure, the pair ofnon-deviating dispersive elements is two double Amici prisms alignedalong the optical axis. The dispersion of the two double Amici prismsmay add up to the total dispersion of the optical beam by the opticalsystem.

In some embodiments, at least one of the double Amici prisms isrotatable relative to each other around the optical axis. In otherembodiments, both double Amici prisms may be independently rotatablearound the optical axis. A rotational angle between the two double Amiciprisms relative to each other around the optical axis may becontinuously adjusted from about 0° to about 180°.

Advantageously, adjusting the rotational angle between the first andsecond double Amici prisms vary the total dispersion of the optical beamby the optical system. This eliminates the need to change the footprintof an optical setup in which the optical system is implemented andfurther allows for a compact design of the optical setup. Additionally,rotational stages for adjusting the rotational angle between the twodouble Amici prisms may operate at a speed faster than that oftranslational stages for adjusting the optical path length between twoprisms or gratings. This further improves the speed for adjustingdispersion of the optical beam.

In some embodiments, adjustment of the rotational angle between the twodouble Amici prisms allows for adjustment of the magnitude of thedispersion of the optical beam. For example, when the rotational anglebetween the two double Amici prisms is about 0°, the dispersion of thetwo prisms add up to a maximum magnitude of dispersion, e.g., doublingthe magnitude of dispersion of one prism. When the rotational anglebetween the two double Amici prisms is about 180°, the dispersion of thetwo prisms may cancel each other, leading to a minimum magnitude ofdispersion, e.g., about zero dispersion. When the rotational anglebetween the two double Amici prisms is an intermediate angle between 0°and 180°, the magnitude of dispersion is between the two extremes. Whenthe two double Amici prisms are identical, the maximum magnitude of thedispersion may double the magnitude of dispersion that can be generatedby one of the double Amici prisms.

In some embodiments, when the rotational angle between the two doubleAmici prisms is continuously adjusted from about 0° to about 180°, themagnitude of the dispersion generated by the optical system may becontinuously adjusted from the maximum magnitude to the minimummagnitude.

According to an aspect of the present disclosure, a predeterminedwavelength, e.g., a center wavelength, of the optical beam would notchange its propagation direction after passing through the opticalsystem. For example, a predetermined wavelength of the input opticalbeam may enter the first double Amici prism along an input optical axis,and then exit the second double Amici prism along an output opticalaxis. The input optical axis and the output optical axis of thepredetermined wavelength may remain collinear. The other wavelengths ofthe optical beam may transmit the optical system with suitable deviationangles that are determined by the design of the prisms.

In some embodiments, the orientation of the dispersion of the opticalbeam caused by the optical system may be adjusted by rotating both ofthe double Amici prisms. As described herein, the orientation of thedispersion may refer to an orientation of a dispersion line formed orfocused on a plane orthogonal to the optical axis after the optical beampasses through the optical system. The dispersion line may have thespectrum of the optical beam spread out along a linear direction.Rotating both prisms may change the angle of incidence of the opticalbeam on the first prism, and thus change the deviation angles for thewavelengths of the optical beam exiting the second prism except for thepredetermined wavelength. The changes of the deviation angles may thenlead to a change of the orientation of the dispersion line.

As described herein, the optical beam entering the optical system to bedispersed may be referred to as an input optical beam, and the dispersedoptical beam exiting the optical system may be referred to as an outputoptical beam. In some embodiments, the output optical beam may befurther modulated, filtered, processed, and/or detected by aone-dimensional or two-dimensional array of photodetector or sensor ofan imaging device.

Reference will now be made in detail to embodiments and aspects of thepresent disclosure, examples of which are illustrated in theaccompanying drawings.

FIG. 28 is a schematic perspective representation of an exemplaryoptical system 2800 for dispersing an optical beam. For example, system2800 may be implemented in an optical setup for generating an outputoptical beam 2802 with a desired dispersion from an input optical beam2801. As described herein, input optical beam 2801 refers to the opticalbeam entering and/or passing through system 2800 and output optical beam2802 refers to the optical beam exiting system 2800. Input optical beam2801 and output optical beam 2802 are referenced separately for thedescribing the transmission and dispersion of the optical beam by system2800. In some embodiments, output optical beam 2802 may be furtherfiltered, modulated, and/or acquired to obtain an optical signal with adesired spectrum and/or spectral resolution.

As shown in FIG. 28, system 2800 may include at least two double Amiciprisms, e.g., prisms 2810 a and 2810 b. Depending on the application ofsystem, system 2800 may further include at least two lenses 2830 a and2830 b. For example, when system 2800 is implemented in a hyperspectralimaging system where emission spectra of a plurality of locations on asample are simultaneously measured, lenses 2830 a and 2830 b may be inan imaging configuration. Lenses 2830 a and 2830 b may create acollimated space between them for input optical beam 2801 to propagatethrough prisms 2810 a and 2810 b. System 2800 may further include otheroptical elements, such as mirrors, beam dumps, spatial filters etc.

As described herein, an optical axis of system 2800 may define a pathalong which a predetermined wavelength (e.g., a center wavelength) ofinput optical beam 2801 and output optical beam 2802 propagates throughsystem 2800.

As shown in FIG. 28, prisms 2810 a and 2810 b and lenses 2830 a and 2830b are aligned along the optical axis of system 2800. Input optical beam2801 may be collected from a focus spot (“O”) in an optical system andcollimated by lens 130 a. For example, spot “O” may be at about onefocal length before lens 2830 a. Collimated input optical beam 2801 maythen propagate through prisms 2810 a and 2810 b. Prisms 2810 a and 2810b may disperse the input optical beam 2801 to a desired magnitude,generating a spectrally dispersed output optical beam 2802 exiting prism2810 b. Additionally or alternatively, prisms 2810 a and 2810 b maychange the orientation of the dispersion of output optical beam 2802.Output optical beam 2802 may be collected and focused by lens 2830 b toa focused spot (“I”). Spot “I” may be at about one focal length afterlens 2830 b. Spot “I” may be in the form of a discrete or continuousspectrum with a desired spread and/or resolution. In some embodiments,spot “I” may be acquired by a sensor array.

Other configurations of system 2800 are possible using additionaloptical elements, such as mirrors, lenses, filters, etc. Although doubleAmici prisms 2810 a and 2810 b are used as examples for thenon-deviating dispersive elements of system 2800, other suitablenon-deviating dispersive elements, such as non-deviating compound prismsmay be used consistent with the embodiments of the present disclosure.

Functions and the working principles of the components of system 2800are described in detail below.

FIG. 29 is a schematic cross-sectional representation of an exemplarydouble Amici prism 2810 a. As described herein, descriptions of thefeatures below in references to prism 2810 a are equally applicable toprism 2810 b.

As shown in FIG. 29, prism 2810 a includes a set of prism elementsplaced in series, such as first element 2812, second element 2814, andthird element 2816. These elements may be cemented together to form asolid assembly. First and second elements 2812 and 2814 may be made ofthe same glass and have the same apex angles. The design layout of prism2810 a is thus symmetric about the plane passing through the center ofthird element 2816.

First and second elements 2812 and 2814 are typically made of a glasshaving lower index of refraction relative to third element 2816. Forexample, first and second elements 2812 and 2814 may be made of crownglass and third element may be made of flint glass. As described herein,the materials of the prism elements may be suitably selected to achievea desired dispersion of input optical beam 2801. In some embodiments,the selection of the materials of the prism elements of prisms 2810 aand 2810 b may be the same such that the dispersion that can begenerated by prisms 2810 a and 2810 b may be the same. In otherembodiments, the selection of the materials of the prism elements ofprisms 2810 a and 2810 b may be different such that the dispersion thatcan be generated by prisms 2810 a and 2810 b may be different.Additionally, the materials of the prism elements may be designed toachieve greater linearity of total dispersion and/or to achievehigher-order dispersion effects of system 2800.

As shown in FIG. 29, prism 2810 a may have two sloping faces on its twosides. A predetermined wavelength of input optical beam 2801 may passthrough the first sloping face of prism 2810 a and exit from the othersloping face with substantially zero deviation from the optical axis.The predetermined wavelength may depend on the design of prism 2810 a,such as the composition of the materials and geometry of the prismelements 2812, 2814, and 2816. Other wavelengths of input optical beam2801 would pass through prism 2810 a with a wavelength-dependentdeviation angle from the optical axis. Such deviation angles may alsodepend on the geometry of prism 2810 a.

Advantageously, as described above, system 2800 may achieve a desireddispersion without causing deviation of the predetermined wavelength,e.g., a center wavelength of input optical beam 2801. In other words,the input optical axis of the predetermined wavelength of input opticalbeam 2801 remains substantially collinear with its output optical axis.

When two prisms 2810 a and 2810 b are used together in system 2800, thedispersion of input optical beam 2801 may be augmented or reduced. FIGS.30 and 31 are graphical cross-sectional illustrations of two examples ofinput optical beam 2801 passing through system 2800. FIG. 30 shows thatprisms 2810 a and 2810 b are aligned around the optical axis andtogether double the magnitude of the total dispersion. In contrast, FIG.31 shows that prisms 2810 a and 2810 b are counter-aligned around theoptical axis and the dispersion generated by prism 2810 a is reduced toa minimum or cancelled by prism 2810 b.

As shown in FIG. 30, prisms 2810 a and 2810 b are aligned with arotational angle of about 0° relative to each other. Two exemplarywavelengths 2801 a and 2801 b of input optical beam 2801 are deflectedat different angles after passing through prism 2810 a. Whilepropagating in the distance from prism 2810 a to prism 2810 b, inputoptical beam 2801 is dispersed and filled with spectral information. Thetwo wavelengths 2801 a and 2801 b of input optical beam 2801 are furtherdeflected and deviated from each other after passing through prism 2810b. Lens 2830 b may then focus the two wavelengths 2801 a and 2801 b inthe output optical beam 2802 to two different spots shifted from eachother, e.g., “I_(A)” and “I_(B),”. If output optical beam 2802 isacquired by a sensor, the focus spots of the two different wavelengths2801 a and 2801 b would end up at two different locations laterallyshifted from each other on the sensor.

In contrast to the example shown in FIG. 30, FIG. 31 shows that prisms2810 a and 2810 b are counter-aligned with a rotational angle of about180° relative to each other. Two exemplary wavelengths 2801 a and 2801 bof input optical beam 2801 are similarly deflected at different anglesafter passing through prism 2810 a as describe above. Input optical beam2801 is thus dispersed in the space between prism 2810 a to prism 2810b. Then, the two wavelengths 2801 a and 2801 b of input optical beam2801 are deflected by prism 2810 b to an opposite direction, therebyreducing or cancelling the dispersion of output optical beam 2802.

As shown in FIG. 31, in some embodiments, when prisms 2810 a and 2810 bare identical, the dispersion generated by prism 2810 a of the twowavelengths 2801 a and 2801 b may be cancelled to zero after passingthrough prism 2810 b. For example, the two wavelengths 2801 a and 2801 bof input optical beam 2801 may be deflected back by prism 2810 b to bealigned with the optical axis. Lens 2830 b may then focus the twowavelengths 2801 a and 2801 b to spots “I_(A)” and “I_(B)” that overlapwith each other. If output optical beam 2802 is acquired by a sensor,the focus spots of the two different wavelengths 2801 a and 2801 b wouldbe acquired at the same location, thereby cancelling the dispersion.

As described above, when prisms 2810 a and 2810 b are aligned with eachother around the optical axis, the dispersion generated by system 2800may be maximized, e.g., double of the amount of dispersion that can begenerated by prism 2810 a. When prisms 2810 a and 2810 b arecounter-aligned with each other around the optical axis, the dispersiongenerated by system 2800 may be minimized, e.g., cancelled to zero whenprism 2810 b is identical to prism 2810 a. By adjusting the rotationalangle between prisms 2810 a and 2810 b around the optical axis fromabout 0° to about 180°, the dispersion of output optical beam 2802 maybe varied to a desired intermediate magnitude between the maximum andminimum.

The rotation of the two prisms 2810 a and 2810 b around the optical axismay be achieved using any suitable rotating device, such as a steppermotor rotary stage or a thermal motor rotary stage. In some embodiments,only one of the prisms 2810 a and 2810 b may be rotated to adjust thedispersion of output optical beam 2802. In other embodiments, bothprisms 2810 a and 2810 b may be rotated to adjust the dispersion ofoutput optical beam 2802.

As described herein, the two wavelengths 2801 a and 2801 b of inputoptical beam 2801 shown in FIG. 30 are exemplary only. Multiple orinfinite wavelengths of input optical beam 2801 may pass through system2800.

As described above, output optical beam 2802 may be acquired or detectedby a sensor array. For example, a light emitting spot emitting inputoptical beam 2801 may become multiple spots of output optical beam 2802after passing through system 2800. The multiple spots of output opticalbeam 2802 acquired on the sensor may be laterally shifted from eachother along a virtual dispersion line. In some situations, if inputoptical beam 2801 has a continuous spectrum, a light emitting spot wouldbe acquired as a spectrally dispersed line on the sensor. The lightintensity of a given spot or at a given location along the dispersedline would correspond to the intensity provided by the correspondingwavelength. Hence, the intensity versus distance along the dispersionline can be transformed to a relationship between the intensity and thewavelength.

FIGS. 32 and 33 are diagrams of optical simulation results of dispersingan optical beam with three wavelengths (e.g., 500 nm, 600 nm, and 650nm) by system 2800. As shown in FIGS. 32 and 33, focus spots of thethree wavelengths are vertically shifted from each other along a virtualdispersion line. The spot of wavelength 550 nm is at located at thecenter of the diagram, the spot of wavelength 600 nm is located belowthat of wavelength 550 nm, and the spot of wavelength 650 nm is locatedbelow that of wavelength 600 nm.

In some embodiments, the spacing between the spots may depend on themagnitude of dispersion of system 2800. For example, a dispersionmagnitude of system 2800 for generating the simulation result shown inFIG. 32 is adjusted to be greater than that for generating thesimulation result shown in FIG. 33. Accordingly, the spacing between thespots of the three wavelengths is greater in FIG. 32 than that in FIG.33.

Advantageously, the adjustment of the dispersion magnitude may allow anybandwidth of input optical beam 2801 to suitably fill a desired space ofa sensor. This may improve the efficiency of using the space on thesensor and may further improve the measurement throughput of an opticalmeasurement system.

For example, in fluorescence spectroscopy or microscopy, system 2800 maybe used to increase the magnitude of dispersion of a desired range offluorescence emission spectrum of fluorophores. This can increase thespectral resolution of the desired range, thereby providing moreinformation of the fluorophores or fluorescent molecules in a sample. Inother instances, such as in multi-spot hyperspectral imaging systems,system 2800 may be used to reduce the dispersion magnitude of sparsefluorescence emission spectra. This allows for more fluorescenceemission spectra to be measured simultaneously, thereby increasing theefficiency of using the sensor space and the measurement throughput ofthe hyperspectral imaging system.

In some embodiments, prisms 2810 a and 2810 b may be designed to achievelinear dispersion of output optical beam 2802. In such instances, thedistance between the focus spots of a given wavelength and a centerwavelength along the dispersion line is linearly proportional to thedifference between the given wavelength and the center wavelength. Inother embodiments, prisms 2810 a and 2810 b may be designed to achievenonlinear dispersion of output optical beam 2802. In such instances, thedeviation angle of a given wavelength from the optical axis of thecenter wavelength may be proportional to the difference between thegiven wavelength and the center wavelength.

As described above, in some embodiments, prisms 2810 a and 2810 b may beadjusted to change the orientation of dispersion of output optical beam2802. For example, the orientation of dispersion or the dispersion linealong which the focus spots are aligned as shown in FIGS. 32 and 33 arealong the vertical direction. When both prisms 2810 a and 2810 b arerotated together to a given angle around the optical axis, theorientation of dispersion or the dispersion line may be adjusted to adifferent direction. For example, if both prisms 2810 a and 2810 b aretogether further rotated by about 90°, the orientation of dispersion orthe dispersion line along which the focus spots are aligned as shown inFIGS. 32 and 33 may be along a horizontal direction.

System 2800 as described herein may be utilized in a variety of methodsand devices for dispersing an optical beam. FIG. 34 is a flowchart of anexemplary method 3400 for dispersing of an optical beam. Method 3400uses system 2800 and features of the embodiments of system 2800described above in reference to FIGS. 28-31.

At step 3402, an optical beam may be received by system 2800. Dependingon the application of system 2800 and/or the collimation status of theoptical beam, the optical beam may be collected and collimated by lens2830 a before dispersion and focused by lens 2830 b after dispersion. Atstep 3404, the optical beam may be transmitted through a pair of doubleAmici prisms 2810 a and 2810 b aligned along the optical axis. At step3406, the optical beam may be dispersed to a predetermined magnitude byrotating at least one of prisms 2810 a and 2810 b relative to the otheraround the optical axis.

Method 3400 may further include additional steps. For example, method3400 may include calibrating system 2800 before rotating the prisms.Other optical components in system 2800, such as lenses 2830 a and 2830b, may be suitably calibrated and aligned such that the input opticalaxis and the output optical axis of a predetermined wavelength of theoptical beam remain collinear.

In some embodiments, method 3400 may further include varying themagnitude of dispersion by adjusting a rotational angle between thefirst and second prisms 2810 a and 2810 b around the optical axis.Additionally or alternatively, method 3400 may further include adjustingthe dispersion of the optical beam to a predetermined orientation byrotating both prisms 2810 a and 2810 b around the optical axis.

X. EXAMPLE IMAGING BY CONVOLVING DISTANCE AND EMITTED LIGHT WAVELENGTH

It can be advantageous to increase the rate at which a sample (e.g., abiological sample) is imaged in order to identify probes in the sample,to detect the location, color, or other properties of fluorophores inthe sample (e.g., fluorophores of such a probe), or to provide someother benefit. Fluorophores of a sample (e.g., of probes in a sample)can be excited over a range of wavelengths known as the excitation band.When they relax to the ground state, the fluorophores can emit light ina wide range of wavelengths known as the emission band. This disclosureincludes embodiments that leverage the wide emission band of afluorophore for simultaneous acquisition of multiple planes in thesample using a modified form of confocal microscopy. These embodimentsmay be combined with other embodiments described herein to increase therate at which a sample may be imaged.

According to an aspect of the present disclosure, an excitation spot issent towards a sample. The excitation spot, according to the presentdisclosure, may be selected such that it excites the sample over allplanes of interest in the axial direction. The excitation optics shouldbe chosen such that the variation of the excitation spot size over theplanes of interest is minimized.

According to an aspect of the present disclosure, the collection opticsof a microscope system intentionally have a large degree of axialchromatic aberration such that different colors (i.e., differentwavelengths) conjugate with an emission or confocal pinhole at differentplanes. Once the light has passed through the pinhole, it can bedispersed with one or more prisms, gratings or other dispersiveelements, so that the spot becomes, on a two-dimensional sensor at theimage plane, a streak or band as different wavelengths are dispersedfrom the prism at different angles, so that the vertical spatial axis ofthe streak contains axial image information. A given pixel location onthe sensor for a given acquisition frame corresponds to a singleemission wavelength, which in turn encodes the fluorescence informationfrom a single volumetric point in the sample.

Advantageously, for each lateral position on a sample, the axialposition of the image information may be encoded by color (i.e.,wavelength).

In some aspects, systems according to the present disclosure maximizethe axial chromatic aberrations in the optics, contrary to the standardpractice of minimizing them. By introducing large focal shift as afunction of wavelength, the chromatic aberrations may be used to encodeaxial information in the emissions. In this way, the information densityon the image sensor can be greatly increased, and fast volumetricimaging may be advantageously realized.

Embodiments of the present disclosure may be also implemented using aconfocal microscope having one or more two-dimensional image sensors. Incontrast to using a conventional achromatic objective, microscopesystems consistent with the disclosed embodiments may include anobjective that is specifically engineered for chromatic aberration asdiscussed herein. Advantageously, these objectives may be considerablycheaper to fabricate than objectives that are designed for minimalchromatic aberration as a result of the larger optical design space.

In certain aspects, dispersion elements may be added in the collectionpath in microscope systems where hyperspectral capabilities are notrequired.

In certain aspects, chromatic aberrations may be introduced in theoptical path outside of the objective. For example, a dispersive tubelens may be used as the axial chromatic element. The chromaticaberration could also be divided among several optical elements (e.g.,both the objective and the tube lens). This may allow flexible selectionof the objective and/or the tube lens, modification of the degrees ofchromatic aberration in the microscopy system, and/or may furthersimplify the system or reduce the cost of the system.

Consistent with embodiments of the present disclosure, the excitationlight may be made to use the chromatic aberrations to generate a verythin excitation beam. As with some embodiments of light sheet imaging,the excitation light could be made to use a Bessel beam or multiplesmall Bessel beams such that, instead of point excitation at the sample,line excitation is used and dispersive elements are used to convert theline(s) to a rectangle(s) on the two-dimensional sensor.

In further exemplary embodiments, digital micromirror devices or spatiallight modulators (SLMs) could be used as artificial pinholes.

Reference will now be made in detail to embodiments and aspects of thepresent disclosure, examples of which are illustrated in theaccompanying drawings. Where possible, the same reference numbers willbe used throughout the drawings to refer to the same or like parts.

FIG. 35 illustrates an exemplary embodiment in schematic form of amicroscope system 3510. This schematic of FIG. 35 is to be understood asnot being drawn to scale. In some embodiments, the microscope 3510 is afluorescence microscope, and more specifically a modified form ofconfocal fluorescence microscope. Embodiments of the present disclosureare applicable to other microscopy techniques, such as stimulatedemission depletion (STED) microscopy for example.

As shown in FIG. 35, system 3510 creates a magnified image of a sample3512 using fluorescence and/or phosphorescence through principles ofoptical sectioning, which are discussed further below. In anillustrative embodiment, the sample 3512 is stained or dyed with afluorophore compound, which absorbs light energy of a specificwavelength (i.e., excitation band) and re-emits light at a differentwavelength (i.e., emission band). The difference between the excitationpeak wavelength and the emission peak wavelength corresponds to theStokes shift.

Various fluorophores may be used, including those known in the art. Aswill be appreciated, fluorophores have varying properties rendering themmore or less useful for a given microscopic application. Excitationbands range from the ultraviolet through the visible spectra, andemission bands typically range from visible light into the near infraredregion. New fluorophores may also offer various combinations ofproperties, both optical and chemical. In some embodiments, fluorophoresmay be linked, where a first fluorophore's emission is quenched by acompanion fluorophore in a process known as fluorescence resonanceenergy transfer (FRET), allowing a different emission wavelength to beachieved.

Referring again to FIG. 35, sample 3512 is depicted having twoillustrative focal planes. In an illustrative embodiment, the firstfocal plane A and the second focal plane B are generally parallel to oneanother and perpendicular to the major optical axis of the microscopesystem 3510. Other geometries are possible using optical elements suchas lenses, mirrors, etc. Objective 3514 is an optical element thatgathers light (visible or otherwise) from the sample. In exemplaryembodiments, the objective 3514 is also used to project excitationradiation upon the sample 3512. In an exemplary embodiment, objective3514 includes a chromatic lens as discussed below with reference to FIG.36.

Dichroic filter 3516 is an optical splitter element employed to permitexcitation radiation 3520 from an illumination source 3518 to pass intothe objective 3514 for projection onto the sample 3512 (shown at 3520′in FIG. 35). The projected excitation radiation 3520′ can take the formof a spot, which can be of varying form, e.g., circular or elliptical.The sample 3512 is penetrated by excitation radiation 3520′ throughmultiple optical planes, for illustration planes A and B, and thefluorophores are concomitantly excited. The excited fluorophores ofsample 3512 will subsequently emit radiation in an emission band, whichin an illustrative embodiment can be across a range of wavelengths orhave a plurality of wavelengths. The dichroic filter 3516 permitsfluorophore emissions 3522 to pass through at 3522′ while rejectingother wavelengths, such as the excitation radiation. In an illustrativeembodiment, the fluorophore emissions 3522 pass through the objective3514, but other optical paths are possible.

Fluorophore emissions 3522′, rendered substantially parallel by theobjective 3514, pass into a tube lens 3524, in an exemplary embodiment.The tube lens 3524 brings the parallel wave trains 3522′ from theobjective 3514 originating at the focal planes of interest, e.g., focalplanes A and B, into convergence at a confocal pinhole 3526.Out-of-focus emissions do not pass through the confocal pinhole 3526 andare eliminated from the image. The focused fluorophore emission wavetrains 3522″ from the tube lens 3524 converge at the confocal pinhole3526, and contain image information from a plurality of focal planes,e.g., focal planes A and B, and the confocal pinhole 3526 can betranslated axially to accommodate the parameters of investigation. In anillustrative embodiment, the objective 3512 is heavily chromatic, asdescribed below with reference to FIG. 36. In addition, a plurality ofpinholes can be employed to increase throughput by obtaining an imagefrom a different lateral position on the sample 3512.

The excitation spot 3520′ can be laterally translated across the sample3512, and can advantageously simultaneously collect images from multipleaxial planes simultaneously based on the chromatic aberrations of thelens. By employing a fluorophore having a wide emission spectrum, imagewave trains at different axial depths in the sample can be encoded bywavelength, as will be discussed in greater detail below.

In an illustrative embodiment, after passing through the confocalpinhole 3526, fluorophore emission wave trains 3522″ can be projectedonto a photomultiplier detector, e.g., a CCD sensor, or an ocular lensto obtain an image. In another illustrative embodiment, once the lighthas passed through the pinhole 3526 it can be dispersed with one or moreprisms or gratings (e.g., prism 3528 in FIG. 35) so that the spot 3530becomes a streak 3532 on a two-dimensional sensor (not shown) at theimage plane 3534. The sensor could be implemented with an sCMOS sensor,although two-dimension silicon APDs arrays and other sensitive sensorscould also be used. For each lateral position on the sample, the axialposition can be encoded by color, which can subsequently beadvantageously encoded onto the pixel number of the sensor. A threedimensional image can be formed by arranging the streaks obtained fromvarious spots at lateral positions on the sample.

Turning to FIG. 36, a schematic representation of an exemplary chromaticlens 3514 a is depicted. Chromatic lens 3514 a is a component lens ofobjective 3514 in illustrative embodiments. A chromatic lens achieves aseparation of various frequencies on the image plane because ofdifferences in the refractive index of the lens at different wavelengthsof incident light. As depicted in FIG. 36, sample 3512 has threeillustrative focal planes indicated at A′, B′ and C′. Fluorophorespresent in sample 3512 may have a relatively broad emission band, suchthat chromatic lens 3514 a can, by virtue of its axial chromatic opticalaberration, focus light from different planes at different wavelengths,as shown in the illustrative embodiment as emission component beams 3522a, 3522 b, and 3522 c. As shown, although these component beamsoriginate at different focal planes A′, B′ and C′, the difference inrefraction of the component beams by the chromatic lens 3514 a, byvirtue of their different wavelengths, allows the component beams toconjugate for transmission ultimately toward the image plane.

In accordance with another aspect of the present disclosure, principlesof polarization can be applied to result in polarized component beams,which can be further processed for additional optical informationdensity.

FIG. 37 is a flowchart of an exemplary method 3700 for simultaneouslyobtaining an image in multiple planes with a microscope system with anaxially chromatic lens. The illustrative method 3700 using the opticalsystem 3512 and features of the embodiments of FIGS. 35 and 36, discussabove.

At step 3702, fluorophores of a sample are excited over a range ofwavelengths over multiple planes of interest throughout the axial depthof the sample. When the fluorophores relax to the ground state, they canemit light in a wide range of wavelengths, which are collected at step3704 using collection optics of the microscope system intentionallyhaving a large degree of axial chromatic aberration. As a result,different colors are conjugated with an emission or confocal pinhole atdifferent planes at step 3706. At step 3708, the light is dispersed withone or more prisms or gratings and the spot becomes a streak on atwo-dimensional sensor at an image plane at step 3710. Image data iscollected from the sensor at step 3712. As discussed above, for eachlateral position on the sample, the axial position will be encoded bycolor, which may be subsequently encoded onto the pixel number of thesensor.

XI. EXAMPLE CONTROLLABLE OPTICAL FILTER

In some applications, it can be advantageous to apply one or morecontrollable bandpasses, highpasses, lowpasses, or other types ofoptical filtering to illumination applied to a sample and/or to imagelight received from such a sample. Such filtering could facilitateimaging of the sample (e.g., of a biological sample) in order toidentify probes in the sample, to detect the location, color, or otherproperties of fluorophores in the sample (e.g., fluorophores of such aprobe), or to provide some other benefit. Example systems providedherein may selectively transmit one or more spectral bands with tunablebandwidths and/or center wavelengths, allowing the generation of opticalbeam(s) with desired spectral bands and/or spectral resolutions.Embodiments of the present disclosure may be implemented in aspectrometer, e.g., an imaging spectrometer, a microscope, e.g., afluorescence microscope, a confocal microscope, a transmissionmicroscope, a reflectance microscope, etc., or a spectral imagingsystem, e.g., a hyperspectral imaging system. Alternatively, embodimentsof the present disclosure may be implemented in a customized imagingsystem built using suitable optical elements.

According to an aspect of the present disclosure, an optical system isprovided for filtering an input optical beam. The input optical beam mayhave a discrete spectrum or a continuous spectrum with a plurality ofwavelengths. The input optical beam may be an excitation light beam forilluminating a sample or an emission light beam collected from a sample.The input optical beam may be filtered by the optical system to generatean output optical beam having selected spectral bands with desiredbandwidths and/or center wavelengths.

According to an aspect of the present disclosure, the optical system mayinclude one or more spectral slicing modules. Each spectral slicingmodule may have a passband that can be flexibly tuned to have a desiredbandwidth and/or a desired center wavelength. In some embodiments, theoptical system may be placed within a collimated beam in an opticalsetup. In other embodiments, the optical system may collimate an inputoptical beam before filtering the input beam and/or may focus an outputoptical beam after such filtering.

According to an aspect of the present disclosure, the optical system maysplit an input optical beam into at least two partial optical beams. Forexample, one or more beamsplitters may be used to split the inputoptical beam into a desired number of partial optical beams. At leastone of the partial optical beams may be directed to transmit through aspectral slicing module. The spectral slicing module may filter thepartial optical beam by transmitting wavelengths within its passband andreflecting wavelengths outside its passband. The optical system may thencombine the partial optical beams, whether filtered or not, into anoutput optical beam using one or more beamsplitters and/or mirrors, forexample.

In some embodiments, the partial optical beams having different spectralbands may be directed through a corresponding number of spectral slicingmodules respectively. Each spectral slicing module may filter thepartial optical beam transmitting through it to a desired spectral band.The optical system may then combine the filtered partial optical beamsinto an output optical beam using one or more beamsplitters and/ormirrors.

According to an aspect of the present disclosure, a beamsplitter forsplitting the input optical beam may be a dichroic beamsplitter thatselectively transmits and reflects light on the basis of wavelength. Forexample, an input optical beam incident on the dichroic beamsplitter maybe spectrally split into two partial optical beams having two differentspectral bands divided around a cut-off wavelength. One partial opticalbeam may transmit through the dichroic beamsplitter and the other mayreflect off from the dichroic beamsplitter.

In some embodiments, the dichroic beamsplitter may have a passband(spectral region of high transmission/low reflectivity), a stopband(spectral region of low transmission/high reflectivity), and atransition region (the spectral region between the passband andstopband). The transition region may be defined as the region betweentwo wavelengths, e.g., a first wavelength at about 90% and a secondwavelength at about 10% peak transmission respectively. A cut-offwavelength at about 50% peak transmission may be at the center of thetransition region.

According to an aspect of the present disclosure, a beamsplitter forcombining two partial optical beams may allow the two partial opticalbeams to propagate along a common optical path after the combination.For example, a beamsplitter for combining the partial optical beams maybe a dichroic beamsplitter that selectively transmits and reflects lighton the basis of wavelength. One partial optical beam may transmitthrough the dichroic beamsplitter along its optical path and the othermay reflect off from the dichroic beamsplitter to propagate along thesame optical path.

In certain aspects, the beamsplitter for combining two partial opticalbeams into an output optical beam may have the same spectralcharacteristics as those of the beamsplitter for splitting the inputoptical beam into the two partial optical beams. For example, the twobeamsplitters may be identical dichroic beamsplitters that reflect andtransmit light based on the same cut-off wavelength. Advantageously,using identical dichroic beamsplitters for the splitting and combiningallows for high transmission and high reflection of the two spectralbands of the two partial optical beams. This further allows forefficient direction and/or collection of the different partial opticalbeams split from the input optical beam to the combined output opticalbeam, thereby reducing loss of light.

According to an aspect of the present disclosure, the spectral slicingmodules may each operate as a bandpass filter with a tunable passband.The bandwidth and/or center wavelength of the passband of each spectralslicing module may be independently adjustable to desired values. Insome embodiments, each spectral slicing module may include a longpassfilter and a shortpass filter aligned along its optical axis. Thelongpass filter and shortpass filter are independently rotatablerelative to the optical axis. Rotating either of the filters may changethe angle of incidence (AOI) of the partial optical beam upon the filterand thus shift the absorption or reflection edge, e.g., cut-offwavelength. For example, increasing the AOI from normal incidence tohigher angles may shift the spectral transmission of the longpass filterand/or shortpass filter towards shorter wavelengths. Thus, the passbandof each spectral slicing module (e.g., the bandwidth and/or centerwavelength) may be tuned by rotating at least one of its longpass and/orshortpass filters relative to the optical axis.

Advantageously, the passband of each spectral slicing module varies as afunction of the AOI upon the longpass and/or shortpass filters withoutexhibiting substantial change in the shape of the spectrum, thepercentage transmission, and/or the out-of-band rejection. Additionally,the bandwidth and/or center wavelength of the passband of each spectralslicing module may be continuously tuned over an entire possiblepassband by changing the AOI of the partial optical beam upon thefilters. Further, by using a series of spectral slicing modules withdifferent passbands, the spectrum of the input optical beam may beselectively filtered to have spectral bands with desired bandwidths andcenter wavelengths.

In certain aspects, the spectral slicing modules may have a series ofpassbands spectrally shifted from one another with overlapping regionsbetween two adjacent passbands. For example, two different spectralslicing modules may have two different passbands for filtering twopartial optical beams. The two passbands may have an overlapping region,and a first passband may span across wavelengths generally longer thanthe second passband. In such instances, the transition region of thedichroic beamsplitter for splitting an input optical beam into the twopartial optical beams may fall within this overlapping region.Advantageously, such characteristics of the dichroic beamsplitter andthe spectral slicing modules reduce potential artifacts that may resultfrom the spectral splitting of the input optical beam and separatefiltering of the partial optical beams.

In certain aspects, at least one of the spectral slicing modules mayfurther include a blocking filter that additionally blocks wavelengthsoutside of the passband of the spectral slicing module. For example, theblocking filter may be a bandpass filter that substantially blocks orrejects wavelengths beyond the passband formed by the longpass andshortpass filters, thereby reducing or eliminating spectralirregularities beyond the passband.

In certain aspects, the optical system may further include one or moremirrors configured to direct the propagation of the input optical beam,the partial optical beams split from the input optical beam, and/or theoutput optical beam. In some embodiments, a pair of mirrors may beconfigured to align a partial optical beam along the optical axis of aspectral slicing module. For example, a first mirror may receive thepartial optical beam and direct it through the components of thespectral slicing module, e.g., longpass and shortpass filters. A secondmirror may receive the filtered partial optical beam, and may furtherdirect it towards a beamsplitter to be combined with another partialoptical beam. The two mirrors may be independently and suitably adjustedto align the propagation of the partial optical beam along the opticalaxis of the spectral slicing module. Advantageously, aligning thedifferent partial optical beams along the optical axes of the spectralslicing modules respectively may eventually allow the partial opticalbeams to propagate along the same direction or the same optical pathafter they are combined.

In certain aspects, a spectral slicing module may further include acompensation filter that realigns the partial optical beam when it islaterally deviated from the optical axis after transmitting through thelongpass and/or shortpass filters. For example, a partial optical beamtransmitting through the longpass and/or shortpass filters at anon-normal AOI may have a lateral displacement from the optical axis ofthe spectral slicing module. The compensation filter may correct thelateral displacement and realign the input optical axis and the outputoptical axis of the spectral slicing module.

In some embodiments, the output optical beam may propagate along thesame direction as the input optical beam does. For example, the inputoptical beam and the output optical beam of the system may remaincollinear, thereby advantageously maintaining the direction of theoverall optical axis of the optical system.

In certain aspects, a spectral slicing module may further include anoptical spatial compensator that adds optical path length to the partialoptical beam transmitting through it. For example, a first spectralslicing module may have an optical path length (OPL) longer than asecond spectral slicing module. Therefore, an optical path difference(OPD) may exist between a first partial optical beam traveling throughthe first spectral slicing module and a second partial optical beamtraveling through the second spectral slicing module. In such instances,the second spectral slicing module may include an optical spatialcompensator, e.g., a glass plate, that adds OPL traveled by the secondpartial optical beam. Advantageously, the addition of the opticalspatial compensator reduces the OPD between the two partial opticalbeams when they are combined in the output optical beam. This mayfurther reduce or eliminate undesirable optical effects, e.g.,interference, that may result from a OPD between the two partial opticalbeams.

As described herein, the optical beam entering the optical system to befiltered may be referred to as an input optical beam, and the filteredoptical beam exiting the optical system may be referred to as an outputoptical beam. In some embodiments, the output optical beam may befurther dispersed, modulated, filtered, processed, and/or detected by aone-dimensional or two-dimensional array of photodetector or sensor ofan imaging device.

Reference will now be made in detail to embodiments and aspects of thepresent disclosure, examples of which are illustrated in theaccompanying drawings.

FIG. 38 is a schematic representation of an exemplary system 3800 forfiltering an optical beam. For example, system 3800 may be implementedin an optical setup for generating an output optical beam 3900′ with adesired spectrum from an input optical beam 3900. As described herein,input optical beam 3900 refers to the optical beam entering and/ortransmitting through system 3800 and output optical beam 3900′ refers tothe filtered optical beam exiting system 3800. Input optical beam 3900and output optical beam 3900′ are referenced separately for describingthe transmission and filtering of the optical beam by system 3800. Insome embodiments, output optical beam 3900′ may be further dispersed,filtered, modulated, and/or acquired to obtain an optical signal with adesired spectrum and/or spectral resolution.

As shown in FIG. 38, system 3800 may include one or more spectralslicing modules, e.g., spectral slicing modules 3810A, 3810B, 3810C, and3810D; a first set of beamsplitters, e.g., beamsplitters 3820A, 3822A,and 3824A; and a second set of beamsplitters, e.g., beamsplitters 3820B,3822B, and 3824B. The first set of beamsplitters may be used to split anoptical beam into separate partial optical beams with different spectralbands. For example, beamsplitter 3820A may be a dichroic beamsplitterthat splits input optical beam 3900 at a first cut-off wavelength,generating two partial optical beams 3910 and 3920 with two differentspectral bands. Similarly, beamsplitter 3822A further splits opticalbeam 3910 into two partial optical beams 3912 and 3914 at a secondcut-off wavelength, and beamsplitter 3824A further splits optical beam3920 into two partial optical beams 3922 and 3924 at a third cut-offwavelength. Therefore, the partial optical beams 3912, 3914, 3922, or3924 split from input optical beam 3900 may each have a differentspectral band.

As described herein, splitting input optical beam 3900 into four partialoptical beams 3912, 3914, 3922, and 3924 as shown in FIG. 38 is usedonly by way of example. It is also possible to split input optical beam3900 into a smaller or greater number of partial optical beams asdesired. In that case, it would merely be necessary to provide acorresponding quantity of beamsplitters. For example, beamsplitter 3824Amay be replaced by a mirror such that optical beam 3920 is not furthersplit into additional partial optical beams. Alternatively, additionalbeamsplitters may be added to further split optical beams 3912 and/or3922. It is also possible to block one or more partial optical beams sothat the spectral bands corresponding to those partial optical beams aresubstantially removed from the spectrum of output optical beam 3900′.

As described above, when a beamsplitter is a dichroic beamsplitter, twopartial optical beams split by the beampslitter from an input opticalbeam would have different spectral bands. In other embodiments, abeamsplitter other than a dichroic beamsplitter may be used in system3800. In such instances, two partial optical beams split by thebeamsplitter may have the same spectrum, and may be further separatelyfiltered by transmitting through different spectral slicing modules.

In some embodiments, at least one of the partial optical beams 3912,3914, 3922, or 3924 may be directed through a spectral slicing module.The spectral slicing module then filters the partial optical beamtransmitting through it to a desired spectral band having a desiredbandwidth and/or center wavelength.

For example, as shown in FIG. 38, partial optical beams 3912, 3914,3922, or 3924 may be respectively directed through a different spectralslicing module. The spectral slicing modules 3810A, 3810B, 3810C, and3810D, may each operate as a tunable bandpass filter with an adjustablebandwidth and an adjustable center wavelength. Therefore, the spectralslicing modules may each filter the partial optical beam transmittingthrough it and generate corresponding filtered partial optical beams3912′, 3914′, 3922′, or 3924′, with desired spectral bands.

As described herein, the four exemplary spectral slicing modules 3810A,3810B, 3810C, and 3810D, for respectively filtering the four partialoptical beams are described only by way of example. It is also possibleto use a smaller or greater number of spectral slicing modules, and aselected number of partial optical beams may be selected and directedthrough the spectral slicing modules. In such instances, a correspondingnumber of beamsplitters and/or mirrors may be used in system 3800.

The second set of beamsplitters may be used to combine the filteredpartial optical beams 3912′, 3914′, 3922′, or 3924′ into the outputoptical beam 3900′. For example, beamsplitter 3822B may be a dichroicbeamsplitter that transmits optical beam 3912′ and reflects optical beam3914′, thereby generating a combined optical beam 3910′ with a spectrumcombining the spectral bands of optical beams 3912′ and 3914′.Similarly, beamsplitter 3824B may be a dichroic beamsplitter thattransmits optical beam 3922′ and reflects optical beam 3924′, therebygenerating a combined optical beam 3920′ with a spectrum combining thespectral bands of optical beams 3922′ and 3924′. Beamsplitter 3820B mayalso be a dichroic beamsplitter that further transmits the combinedoptical beam 3910′ and reflects the combined optical beam 3920′, therebygenerating output optical beam 3900′. Therefore, output optical beam3900′ of system 3800 would have a spectrum combining the spectral bandsof optical beams 3912′, 3914′, 3922′, and 3924′.

In some embodiments, the second set of beampslitters may have spectralcharacteristics matching those of the first set of beamsplitters toreduce the loss of light. For example, beamsplitters 3822A and 3822B maybe similar or identical dichroic beamplitters having the same cut-offwavelength. Similarly, beamsplitters 3824A and 3824B may be similar oridentical dichroic beamplitters having the same cut-off wavelength, andbeamsplitters 3820A and 3820B may be similar or identical dichroicbeamplitters having the same cut-off wavelength. This matchingconfiguration of the first and second sets of beamsplitters may allowfor highly efficient transmission and reflection of the partial opticalbeams by reducing the mismatching of the cut-off wavelengths of thesebeamsplitters. Advantageously, this may further increase the efficiencyof directing the partial optical beams split from input optical beam3900 to the combined output optical beam 3900′, thereby reducing loss oflight

In some embodiments, system 3800 may further include one or more mirrorsfor independently aligning the partial optical beams transmittingthrough the corresponding spectral slicing modules. The mirrors may beused in pairs for performing the alignment. For example, as shown inFIG. 38, a first pair of mirrors 3830A and 3830B may be adjusted toalign the direction of optical beam 3914 along the optical axis ofspectral slicing module 3810A. Similarly, a second pair of mirrors 3832Aand 3832B may be adjusted to align the direction of optical beam 3912along the optical axis of spectral slicing module 3810B; a third pair ofmirrors 3834A and 3834B may be adjusted to align the direction ofoptical beam 3924 along the optical axis of the spectral slicing module3810C; and a fourth pair of mirrors 3836A and 3836B may be adjusted toalign the direction of optical beam 3922 along the optical axis of thespectral slicing module 3810D.

In some cases, a pair of mirrors for aligning a partial optical beam,e.g., mirrors 3832A and 3832B, may be separately placed at two ends ofthe corresponding spectral slicing module along its optical axis. Inother cases, a pair of mirrors for aligning a partial optical beam,e.g., mirrors 3834A and 3834B, may be placed at the same end of thecorresponding spectral slicing module along its optical axis. Themirrors of system 3800 may be independently tilted and/or rotatedmanually or using motorized devices. For example, the mirrors may beadjusted using stepper, servo, or DC motorized rotational stages.Alternatively, pairs of mirrors may be replaced with pairs ofgalvanometer scanners or galvo mirrors.

Advantageously, independent alignment of the partial optical beams alongthe optical axes of the spectral slicing modules allows the filteredpartial optical beams to propagate along the same direction or the sameoptical path after they are combined. For example, as shown in FIG. 38,optical beams 3912′ and 3914′ would propagate along the same directionafter being combined into optical beam 3910′ by beamsplitter 3822B.Similarly, optical beams 3922′ and 3924′ would propagate along the samedirection after being combined into optical beam 3920′ by beamsplitter3824B, and optical beams 3910′ and 3920′ would then propagate along thesame direction after being further combined into output optical beam3900′ by beamsplitter 3820B.

Functions and the working principles of the spectral slicing modules ofsystem 3800 are described in detail below.

FIG. 39 is a schematic representation of an exemplary spectral slicingmodule for filtering an optical beam. As described herein, descriptionsof the features below in references to spectral slicing module 3810A areequally applicable to other spectral slicing modules of system 3800,e.g., spectral slicing modules 3810B, 3810C, and 3810D.

As shown in FIG. 39, spectral slicing module 3810A may include alongpass filter 3812 and a shortpass filter 3814 aligned along itsoptical axis. Longpass filter 3812 and shortpass filter 3814 may incombination form a bandpass filter with a passband delineated by theiredge wavelengths or cut-off wavelengths (the cut-off wavelength oflongpass filter 3812 is smaller than that of shortpass filter 3814). Atleast one of the longpass filter 3812 and shortpass filter 3814 isrotatable relative to the optical axis. For example, longpass filter3812 and shortpass filter 3814 may be independently rotatable to be atan angle relative to the optical axis.

In some embodiments, longpass filter 3812 and shortpass filter 3814 maybe thin-film angle-tuning filters. Rotating longpass filter 3812 mayadjust the angle of incidence (AOI) of optical beam 3914 upon itssurface. The cut-off wavelength of longpass filter 3812 may vary as afunction of the AOI. Similarly, rotating shortpass filter 3814 mayadjust the AOI of optical beam 3914 upon its surface and the cut-offwavelength of shortpass filter 3814 may vary as a function of the AOI.

For example, rotating longpass filter 3812 to change the AOI of opticalbeam 3914 upon its surface from normal incidence to higher angles mayshift the cut-off wavelength of longpass filter 3812 towards shorterwavelengths. Alternatively, rotating longpass filter 3812 to change theAOI from higher angles to normal incidence may shift the cut-offwavelength of longpass filter 3812 towards longer wavelengths.Similarly, rotating shortpass filter 3814 to change the AOI of opticalbeam 3914 upon its surface from normal incidence to higher angles mayshift the cut-off wavelength of shortpass filter 3814 towards shorterwavelengths. Rotating shortpass filter 3814 to change the AOI fromhigher angles to normal incidence may shift the cut-off wavelength ofshortpass filter 3814 towards longer wavelengths.

Accordingly, tuning the AOI of optical beam 3914 upon longpass filter3812 and/or upon shortpass filter 3814 varies the cut-off wavelengths ofthe passband of spectral slicing module 3810A, thereby allowing foradjustment of the bandwidth and/or center wavelength of the passband.The AOI of optical beam 3914 upon longpass filter 3812 and/or shortpassfilter 3814 may be independently and continuously tuned across a givenrange of adjustment, e.g., from about −10° to about 60°. This mayadvantageously allow the passband of spectral slicing module 3810A to becontinuously tuned to have any desired bandwidth and/or centerwavelength across a given spectral range that could be provided by thefilters.

As described herein, the order of optical beam 3914 transmitting throughlongpass filter 3812 and shortpass filter 3814 would not affect thetunable bandpass filtering of optical beam 3914 by spectral slicingmodule 3810A. This also applies to the other spectral slicing modulesfor filtering other partial optical beams in system 3800.

Comparing to a single tunable bandpass filter whose predeterminedpassband may be shifted by tuning the AOI of the optical beam on thefilter, spectral slicing module 3810A advantageously allows for flexibleadjustment of the bandwidth and/or the center wavelength of its passbandby independently tuning the two cut-off wavelengths of the passband.Additionally, comparing to other tunable optical filters, such as liquidcrystal tunable filters (LCTF), acousto-optic tunable filters (AOTF), orlinear variable tunable filters (LVTF), spectral slicing module 3810Aallow for high transmission, sharp cut-off edges, and polarizationinsensitivity provided by the longpass and shortpass filters.

In some situations, when optical beam 3914 transmits through longpassfilter 3812 and/or shortpass filter 3814 at non-normal angles, filteredoptical beam 3914′ may laterally deviate from the optical axis ofspectral slicing module 3810A. In such situations, as shown in FIG. 39,spectral slicing module 3810A may further include a compensation filter3816 aligned along its optical axis after longpass filter 3812 andshortpass filter 3814. Compensation filter 3816 may be rotated to be ata suitable angle relative to the optical axis to generate an oppositelateral deviation to correct the lateral displacement of optical beam3914′. In some embodiments, compensation filter 3816 may be adjusted tobe at an angle relative to the optical axis ranging from about 0° toabout 30°.

In some embodiments, compensation filter 3816 may be adjusted togetherwith mirrors to align the filtered optical beam 3914′ along the opticalaxis of spectral slicing module 3810A. For example, mirrors 3830A and3830B and compensation filter 3816 may be independently adjusted toallow optical beam 3914′ to propagate along the optical axis of spectralslicing module 3810A. Similar alignment may be performed for otheroptical beams, e.g. optical beams 3912′, 3922′, and 3924′. Additionally,as shown in FIG. 38, such independent alignment of the filtered partialoptical beams may further allow them to propagate along the same opticalpath after they are combined into one optical beam (e.g., optical beams3910′, 3920′, and 3900′).

In some embodiments, longpass filter 3812, shortpass filter 3814, and/orcompensation filter 3816 may be independently rotated using motorizedrotational devices. For example, these filters may be adjusted usingstepper, servo, or DC motorized rotational stages. Alternatively, thesefilters may be rotated using galvanometer scanners.

In some situations, separate optical beams, e.g., optical beams 3912 and3914, may propagate through different optical path lengths (OPL). Forexample, as shown in FIG. 38, due to the geometry of system 3800,optical beam 3912′ may propagate through an OPL longer than that ofoptical beam 3914′ when they are combined at beamsplitter 3822B. Theoptical path length difference (OPD) between optical beams 3912′ and3914′ may result in a phase shift between them when they are combined.In some instances, this phase shift may generate undesirable opticaleffects, e.g., interference. Therefore, in some embodiments, spectralslicing module 3810A may further include an optical spatial compensator3818.

Optical spatial compensator 3818 may add OPL to the optical beamtransmitting through it. Optical spatial compensator 3818 may be a glassplate with a selected thickness and refractive index. For example, asshown in FIG. 38, optical spatial compensator 3818 may add to the OPLtraveled by optical beam 3914′ to be the same as that traveled byoptical beam 3912′. Advantageously, the addition of optical spatialcompensator 3818 may allow two partial optical beams transmitting alongtwo different paths in system 3800 to propagate through the same amountof OPL upon being combined into one optical beam, thereby reducing oreliminating undesirable optical effects.

In some embodiments, spectral slicing module 3810A may further include ablocking filter (not shown) that further blocks or rejects wavelengthsoutside of a desired passband of the spectral slicing module 3810A. Forexample, the blocking filter may be a bandpass filter that substantiallyblocks or rejects wavelengths beyond the passband formed by longpassfilter 3812 and shortpass filter 3814. This advantageously allows forreducing or eliminating potential non-ideal spectral irregularitiesbeyond the passband. Additionally or alternatively, spectral slicingmodule 3810A may further include another compensation filter (not shown)that compensates for astigmatism to improve the sharpness of thefiltered optical beam 3914′. This may further improve the sharpness ofoutput optical beam 3900′.

As described above, the spectral slicing modules may have a series ofpassbands spectrally shifted from one another with overlapping regionsbetween two adjacent passbands. In such instances, the transition regionof the dichroic beamsplitter for splitting an input optical beam intothe two partial optical beams is selected to be within the overlappingregion to reduce potential artifacts from the splitting and separatefiltering of the partial optical beams. FIGS. 40A-40C graphicallyillustrate the advantage of such characteristics of the dichroicbeamsplitter and the spectral slicing modules for reducing loss oflight. FIGS. 40A-40C show two exemplary passbands 4040A and 4040B ofspectral slicing modules 3810A and 3810B with amount of transmission(e.g., percentage transmission) along the vertical axis and wavelengthalong the horizontal axis. Additionally, FIGS. 40A-40C show an exemplarytransmission spectrum 4050 of beamsplitter 3822A having a transitionregion 4030.

As shown in FIG. 38 and FIG. 40A, passbands 4040A and 4040B do notoverlap. In such instances, wavelengths of optical beam 3910 at thecenter of transition region 4030 partially transmit through and reflectfrom beamsplitter 3822A. However, because passbands 4040A and 4040B donot overlap, and may have gap regions due to the slopes of their edges,at least a portion of optical beam 3910 at these wavelengths at thecenter of transition region 4030 cannot pass through either passband4040A or 4040B. This results in losing a portion of optical beam 3910 atthese wavelengths. Further, wavelengths of optical beam 3910 at the twoedges of transition region 4030 may be subjected to additional loss. Forexample, some wavelengths of optical beam 3910 in transition region 4030may substantially transmit through beamsplitter 3822A and then throughpassband 4040B of spectral slicing module 3810B. But these wavelengthsof optical beam 3910 are also partially reflected from beamsplitter3822A and directed towards spectral slicing module 3810A. However, thesewavelengths do not fall in passband 4040A of spectral slicing module3810A, thereby resulting in losing a portion of optical beam 3910 atthese wavelengths.

As shown in FIG. 38 and FIG. 40B, passbands 4040A and 4040B have anoverlapping region 4040C narrower than transition region 4030 ofbeamsplitter 3822A. In such instances, the wavelengths of optical beam3910 at the center of transition region 4030 would transmit throughpassbands 4040A and 4040B, thereby reducing loss of optical beam 3910.However, wavelengths of optical beam 3910 at the two edges of transitionregion 4030 that are outside of overlapping region 4040C may still besubjected to additional loss. For example, some wavelengths of opticalbeam 3910 outside overlapping region 4040 but inside transition region4030 may substantially transmit through beamsplitter 3822A and thenthrough passband 4040B of spectral slicing module 3810B. But thesewavelengths of optical beam 3910 are also partially reflected frombeamsplitter 3822A and directed towards spectral slicing module 3810A.However, these wavelengths do not fall in passband 4040A of spectralslicing module 3810A. This again results in losing a portion of opticalbeam 3910 at these wavelengths.

As shown in FIG. 38 and FIG. 40C, according to embodiments of thepresent disclosure, passbands 4040A and 4040B have an overlapping region4040C equal to or wider than transition region 4030 of beamsplitter3822A. In such instances, wavelengths of optical beam 3910 in transitionregion 4030 would fall in both passbands 4040A and 4040B. This allowsfor portions of optical beam 3910, whether reflecting from ortransmitting through beamsplitter 3822A, to transmit through passband4040A of spectral slicing module 3810A and/or passband 4040B of spectralslicing module 3810B, thereby advantageously reducing or eliminating ofthe loss of optical beam 3910.

As described herein, the adjustment of the mirrors and angle-tuning ofthe filters of system 3800 may be controlled by a controller (notshown). The controller may have a processor, a non-transitory memory,and a computer-readable medium that stores instructions or operationalsteps. The memory may store a plurality of coefficients of the filters,such as AOI and cut-off wavelengths, and parameters of the mirrors,e.g., angles relative to the optical axis along one or two spatialdimensions. The instructions or steps, when executed by the processor,may adjust the AOI of the optical beams upon the filters to suitableangles based on the desired passbands of the spectral slicing modules.Additionally, the instructions or steps, when executed by the processor,may further operate motorized rotational stages or galvanometer scannersto adjust the mirrors and/or compensation filters to align the outputpartial optical beams along the spectral slicing modules such that theywould propagate along the same optical path after being combined.

Examples for filtering input optical beam 3900 by system 3800 togenerate output optical beam 3900′ with desired spectral bands arefurther described below in reference to their spectra. As describedabove, an input optical beam 3900 may be split into a plurality ofpartial optical beams having different spectral bands. The spectralbands of the partial optical beams may be selectively and independentlyfiltered to desired spectral ranges by the corresponding spectralslicing modules. When the partial optical beams are combined into outputoptical beam 3900′, the spectral bands of the partial optical beams arethen combined as the spectrum of output optical beam 3900′.

FIG. 41 is a graphical illustration for an exemplary spectrum 4000 ofinput optical beam 3900. FIGS. 42A and 42B are graphical illustrationsfor examples of the spectrum 4000′ of output optical beam 3900′.

As shown in FIG. 41, spectrum 4000 of input optical beam 3900 may besplit into four spectral bands 4012, 4014, 4022, and 4024, correspondingto the four partial optical beams 3912, 3914, 3922, and 3924. Forexample, spectrum 4000 of input optical beam 3900 may first be splitinto spectra 4010 and 4020, corresponding to optical beams 3910 and3920. Spectrum 4010 of optical beam 3910 may then be further split intospectral bands 4012 and 4014, corresponding to optical beams 3912 and3914. Similarly, spectrum 4020 of optical beam 3920 may then be furthersplit into spectral bands 4022 and 4024, corresponding to optical beams3922 and 3924. In some embodiments, as shown in FIG. 41, adjacentspectral bands may have overlapping bands due to the transition regions4030 of the beamsplitters.

As shown in FIGS. 42A and 42B, spectral bands 4012, 4014, 4022, and 4024may be filtered by the corresponding spectral slicing modules to desiredspectral bands 4012′, 4014′, 4022′, and 4024′, corresponding to thefiltered optical beams 3912′, 3914′, 3922′, and 3924′. Spectrum 4000′ ofoutput optical beam 3900′ is the combination of the filtered spectralbands.

In one example, as shown in FIG. 42A, spectral band 4012 of optical beam3912 may be filtered to a narrower spectrum 4012′ with a centerwavelength λ_(a). Additionally or alternatively, as shown in FIG. 42B,spectral slicing module 3810A may be adjusted to tune the centerwavelength λ_(a) of spectral band 4012′ towards shorter wavelengths asdesired. Spectral slicing module 3810A may also be adjusted to tune thecenter wavelength λ_(a) of spectral band 4012′ towards longerwavelengths as needed (not shown).

In another example, as shown in FIG. 42A, spectral band 4014 of opticalbeam 3914 may be filtered to a desired spectral band 4014′ with a centerwavelength λ_(b). Additionally or alternatively, as shown in FIG. 42B,spectral slicing module 3810B may be adjusted to reduce the bandwidth ofspectral band 4014′ and to shift the center wavelength λ_(b) of spectralband 4014′ towards longer wavelengths as desired.

As shown in FIG. 42B, spectral band 4012′ of filtered optical beams3912′ and spectral band 4014′ of filtered optical beams 3914′ can besubstantially continuous, maintaining the continuity of spectra bands4012 and 4014 of input optical beam 3900 without substantial loss or noloss of light. This may be advantageously achieved by selectively usingbeamsplitter 3822A with a transition region 4030 equal to or narrowerthan that of overlapping region 4040C of the passbands of spectralslicing modules 3810A and 3810B as described above.

In another example, as shown in FIG. 42A, spectral band 4024 of opticalbeam 3924 may be filtered to a desired spectral band 4024′ with a centerwavelength λ_(c). Additionally or alternatively, as shown in FIG. 42B,spectral slicing module 3810C may be adjusted to increase the bandwidthof spectral band 4024′. Spectral slicing module 3810C may also beadjusted to shift the center wavelength λ_(c) of spectral band 4024′towards longer or shorter values (not shown).

In yet another example, as shown in FIG. 42A, spectral band 4022 ofoptical beam 3922 may be filtered to a desired spectral band 4022′ witha center wavelength λ_(d). Additionally or alternatively, as shown inFIG. 42B, spectral slicing module 3810D may be adjusted to increase thebandwidth of spectral band 4022′ and tune the center wavelength λ_(d)towards shorter wavelengths as needed. Alternatively, spectral slicingmodule 3810D may be adjusted to reduce the bandwidth of spectral band4022′, and/or tune the center wavelength λ_(d) towards shorter or longerwavelengths as needed (not shown).

As described herein, FIGS. 42A and 42B only provide exemplary tuning ofthe spectral filtering that can be provided by the spectral slicingmodules of system 3800. As described above, any of the spectral slicingmodules may be adjusted to filter a partial optical beam to have anydesired spectral band and center wavelength in a given spectral range.As describe herein, this given spectral range may be determined by thetunable ranges of cut-off wavelengths of the longpass and shortpassfilters of each spectral slicing module.

System 3800 as described herein may be utilized in a variety of methodsand devices for filtering an optical beam. FIG. 43 is a flowchart of anexemplary method 4300 for filtering of an optical beam. Method 4300 usessystem 3800 and features of the embodiments of system 3800 describedabove in reference to FIG. 38-42.

At step 4302, an input optical beam (e.g., optical beam 3910) is splitinto a first optical beam (e.g., optical beam 3914) and a second opticalbeam (e.g., optical beam 3912) using a first beamsplitter (e.g.,beamsplitter 3822A). At step 4304, the first optical beam is filtered bytransmitting the first optical beam through a first spectral slicingmodule (e.g., spectral slicing module 3810A) having a first passband(e.g., passband 4040A). At step 4306, the second optical beam (e.g.,optical beam 3914) is filtered by transmitting the second optical beamthrough a second spectral slicing module (e.g., spectral slicing module3820A) having a second passband (e.g., passband 4040B). At step 4308,the first optical beam may be combined with the second optical beam intoan output optical beam (e.g., optical beam 3910′) using a secondbeamsplitter (e.g., beamsplitter 3822B).

As described herein, an input optical beam may be split into a desirednumber of partial optical beams with different spectral bands using asuitable quantity of beamsplitters. The above-described steps may beperformed for a plurality of times based on the number of spectral bandsdesired to be split and filtered of an input optical beam.

Various embodiments of method 4300 may include one or more of thefollowing features or steps. For example, method 4300 may furtherinclude tuning a bandwidth and/or a center wavelength of the passband ofat least one of the spectral splicing modules by varying the AOI of thepartial optical beam upon its longpass filter 3812 and/or the shortpassfilter 3814. In some embodiments, method 4300 may further includedirecting the propagation of the first and/or second optical beams usingone or more mirrors, e.g., pairs of mirrors. Method 4300 may furtherinclude realigning the first and/or second optical beams laterallydeviated from the optical axis after transmitting through the longpassand/or shortpass filters using one or more rotatable mirrors and/orcompensation filters.

In some embodiments, method 4300 may further include directing the firstand second optical beams to propagate along the same direction oroptical path after they are combined. Additionally, method 4300 mayfurther include directing the combined output optical beam to becollinear with the input optical beam.

In some embodiments, method 4300 may further include additionallyblocking the wavelengths outside of the passband of at least one of thespectral splicing modules using a blocking filter. Method 4300 mayfurther include adding optical path length to the first and/or secondoptical beams using at least one optical spatial compensator.

XII. CONCLUSION

The particular arrangements shown in the Figures should not be viewed aslimiting. It should be understood that other embodiments may includemore or less of each element shown in a given Figure. Further, some ofthe illustrated elements may be combined or omitted. Yet further, anexemplary embodiment may include elements that are not illustrated inthe Figures.

Moreover, it is particularly noted that while devices, systems, methods,and other embodiments are described herein by way of example as beingemployed to image biological environments (e.g., tissues extracted froma human body) and to determine the identity of probes within suchenvironments, it is noted that the disclosed devices, systems, andmethods can be applied in other contexts as well. For example, imagingsystems configured as disclosed herein may be included as part of otherscientific and/or industrial imaging apparatus. Embodiments of thepresent disclosure may be implemented in a spectrometer, e.g., animaging spectrometer, a microscope, e.g., a fluorescence microscope, aconfocal microscope, a transmission microscope, a reflectancemicroscope, etc., or a spectral imaging system, e.g., a hyperspectralimaging system, or in some other imaging system

Additionally, while various aspects and embodiments have been disclosedherein, other aspects and embodiments will be apparent to those skilledin the art. The various aspects and embodiments disclosed herein areincluded for purposes of illustration and are not intended to belimiting, with the true scope and spirit being indicated by thefollowing claims. Other embodiments may be utilized, and other changesmay be made, without departing from the spirit or scope of the subjectmatter presented herein. It will be readily understood that the aspectsof the present disclosure, as generally described herein, andillustrated in the figures, can be arranged, substituted, combined,separated, and designed in a wide variety of different configurations,all of which are contemplated herein.

What is claimed is:
 1. A system comprising: a light sensor thatcomprises a plurality of light-sensitive elements disposed on a focalsurface of the light sensor; a spatial light modulator that comprises areflective layer disposed beneath a refractive layer and that isoperable to have a refractive index that varies spatially across thespatial light modulator according to a controllable gradient, wherein atleast the direction and magnitude of the controllable gradient areelectronically controllable, and wherein the refractive layer ischromatically dispersive; an optical system that (i) directs lightemitted from a target toward the spatial light modulator and (ii)directs light emitted from the target and reflected from the spatiallight modulator to the light sensor such that the focal surface of thelight sensor is conjugate to a focal surface passing through the target;and a controller that is operably coupled to the light sensor and thespatial light modulator and that is operable to perform controlleroperations comprising: controlling the spatial light modulator such thatat least one of the direction or magnitude of the controllable gradientare different during each of a plurality of periods of time; generating,using the light sensor, a plurality of images of the target, whereineach image corresponds to a respective one of the plurality of periodsof time; determining, based on the plurality of images, locations andcolors of two or more fluorophores in the target; and determining, basedon the determined colors and locations of the two or more fluorophores,an identity of a probe that is located in the target and that comprisesthe two or more fluorophores.
 2. The system of claim 1, wherein theoptical system collimates the light emitted from the target that isdirected toward the spatial light modulator.
 3. The system of claim 1,further comprising: a light source; and a micromirror device, whereinthe micromirror device comprises a substantially planar array ofactuatable mirrors disposed on a surface, wherein respective angles ofthe actuatable mirrors relative to the surface are electronicallycontrollable, wherein the optical system directs the light from thelight source to the target via reflection from a first set of one ormore of the actuatable mirrors, and wherein the optical system directsthe light emitted from the target in response to the illumination towardthe spatial light modulator via reflection from the first set of one ormore actuatable mirrors such that the surface of the micromirror deviceis conjugate to the focal surface passing through the target.
 4. Thesystem of claim 1, wherein the spatial light modulator comprises anarray of regions having respective electronically controllablerefractive indexes.
 5. A system comprising: a first light sensor thatcomprises a plurality of light-sensitive elements disposed on a focalsurface of the first light sensor; a second light sensor that comprisesa plurality of light-sensitive elements; a chromatically dispersiveelement; an optical system that (i) directs light emitted from aparticular region of a target to the first light sensor such that thefocal surface of the first light sensor is conjugate to a focal surfacepassing through the particular region of the target, (ii) directs lightemitted from the particular region of the target toward thechromatically dispersive element, and (iii) directs light emitted fromthe particular region of the target that has interacted with thechromatically dispersive element to the second light sensor such thatlight of different wavelengths that is emitted from the particularregion of the target is received by corresponding differentlight-sensitive elements of the second light sensor; and a controllerthat is operably coupled to the first light sensor and the second lightsensor and that is operable to perform controller operations comprising:generating, using the plurality of light-sensitive elements of the firstlight sensor, a first plurality of respective time-varying waveforms oflight emitted from respective different locations of the particularregion of the target; generating, using the plurality of light-sensitiveelements of the second light sensor, a second plurality of respectivetime-varying waveforms of light emitted from the particular region ofthe target at respective different wavelengths; determining correlationsbetween time-varying waveforms of the first plurality of time-varyingwaveforms and time-varying waveforms of the second plurality oftime-varying waveforms; determining, based on the determinedcorrelations, locations and colors of two or more fluorophores in thetarget; and determining, based on the determined colors and locations ofthe two or more fluorophores, an identity of a probe that is located inthe target and that comprises the two or more fluorophores.
 6. Thesystem of claim 5, wherein the chromatically dispersive elementcomprises a spatial light modulator, wherein the spatial light modulatorcomprises a reflective layer disposed beneath a refractive layer,wherein the refractive layer is configured to have a refractive indexthat varies spatially across the spatial light modulator according to acontrollable gradient, wherein at least the direction and magnitude ofthe controllable gradient are electronically controllable, and whereinthe refractive layer is chromatically dispersive.
 7. The system of claim6, wherein the optical system collimates the light emitted from thetarget that is directed toward the spatial light modulator.
 8. Thesystem of claim 6, wherein the spatial light modulator comprises anarray of cells having respective electronically controllable refractiveindexes.
 9. The system of claim 5, further comprising: a light source;and a micromirror device, wherein the micromirror device comprises asubstantially planar array of actuatable mirrors disposed on a surface,wherein respective angles of the actuatable mirrors relative to thesurface are electronically controllable, wherein the optical systemdirects the light from the light source to the particular region of thetarget via reflection from a first set of one or more of the actuatablemirrors, wherein the optical system directs the light emitted from thetarget in response toward the first light sensor via reflection from thefirst set of one or more actuatable mirrors such that the surface of themicromirror device is conjugate to the focal surface passing through theparticular region of the target and such that the focal surface of thefirst light sensor is conjugate to the focal surface passing through theparticular region of the target, and wherein the optical system directsthe light emitted from the target in response to the illumination towardthe chromatically dispersive element via reflection from a second set ofone or more of the actuatable mirrors, and wherein the one or moreactuatable mirrors in the first set have a first angle relative to thesurface of the micromirror device.
 10. The system of claim 5, furthercomprising: a light source; and a micromirror device, wherein themicromirror device comprises a substantially planar array of actuatablemirrors disposed on a surface, wherein respective angles of theactuatable mirrors relative to the surface are electronicallycontrollable, wherein the optical system directs the light from thelight source to the particular region of the target via reflection froma first set of one or more of the actuatable mirrors, wherein theoptical system directs the light emitted from the target in responsetoward the first light sensor and the chromatically dispersive elementvia reflection from the first set of one or more actuatable mirrors suchthat the surface of the micromirror device is conjugate to the focalsurface passing through the particular region of the target and suchthat the focal surface of the first light sensor is conjugate to thefocal surface passing through the particular region of the target. 11.The system of claim 5, further comprising an actuated stage, wherein theactuated stage is operable to control the location of the targetrelative to the optical system.
 12. The system of claim 5, wherein adimension of the particular region of the target is approximately equalto a diffraction limit of the optical system.
 13. A method comprising:generating, using a plurality of light-sensitive elements of a firstlight sensor that are disposed on a focal surface of the first lightsensor, a first plurality of respective time-varying waveforms of lightthat is emitted from respective different locations of a particularregion of a target and transmitted to the light sensor via an opticalsystem, wherein the optical system provides the emitted light from thetarget to the first light sensor such that the focal surface of thefirst light sensor is conjugate to a focal surface passing through theparticular region of the target; generating, using a plurality oflight-sensitive elements of a second light sensor, a second plurality ofrespective time-varying waveforms of light at different respectivewavelengths that is emitted from the particular region of the target andtransmitted to the light sensor via the optical system, wherein theoptical system provides the emitted light from the target to achromatically dispersive element, wherein the optical system providesthe emitted light from the target that has interacted with thechromatically dispersive element to the second light sensor such thatlight of different wavelengths that is emitted from the particularregion of the target is received by corresponding differentlight-sensitive elements of the second light sensor; determiningcorrelations between time-varying waveforms of the first plurality oftime-varying waveforms and time-varying waveforms of the secondplurality of time-varying waveforms; determining, based on thedetermined correlations, locations and colors of two or morefluorophores in the target; and determining, based on the determinedcolors and locations of the two or more fluorophores, an identity of aprobe that is located in the target and that comprises the two or morefluorophores.
 14. The method of claim 13, wherein a distance between thetwo or more fluorophores of the probe is less than approximately 50nanometers.
 15. The method of claim 13, further comprising: determiningcorrelations between different time-varying waveforms of the firstplurality of time-varying waveforms, wherein determining locations oftwo or more fluorophores in the target comprises determining thelocation of a fluorophore in the target based at least in part on thedetermined correlations between different time-varying waveforms of thefirst plurality of time-varying waveforms.
 16. The method of claim 13,wherein determining a color of a fluorophore in the target comprises:determining that a determined correlation between a particular generatedtime-varying waveform of light of the first plurality of time-varyingwaveforms of light and a particular generated time-varying waveform oflight of the second plurality of time-varying waveforms of light isgreater than a threshold, wherein the particular generated time-varyingwaveform of light of the first plurality of time-varying waveforms oflight corresponds to light received from the location of the fluorophorein the target; and determining that the color of the fluorophoreincludes a wavelength of light corresponding to the particulartime-varying waveform of light of the second plurality of time-varyingwaveforms of light.
 17. The method of claim 13, further comprising:generating illumination using a light source; and operating amicromirror device to electronically control respective angles ofactuatable mirrors of the micromirror device relative to a surface ofthe micromirror device, wherein the actuatable mirrors comprise asubstantially planar array and are disposed on the surface of themicromirror device, and wherein operating the micromirror device toelectronically control respective angles of actuatable mirrors of themicromirror device comprises controlling a first set of one or more ofthe actuatable mirrors to have a first angle relative to the surface ofthe micromirror device, and wherein the optical system directs theillumination from the light source to the particular region of thetarget via reflection from the first set of one or more actuatablemirrors, and wherein the optical system directs the light emitted fromthe target in response to the illumination toward the first light sensorvia reflection from the first set of one or more actuatable mirrors suchthat the surface of the micromirror device is conjugate to the focalsurface passing through the particular region of the target.
 18. Themethod of claim 13, wherein the chromatically dispersive elementcomprises a spatial light modulator, the method further comprising:electronically controlling the spatial light modulator such that arefractive layer of the spatial light modulator has a refractive indexthat varies spatially across the spatial light modulator according to acontrollable gradient, wherein the controllable gradient has at least afirst specified direction and a first specified magnitude, wherein thespatial light modulator further comprises a reflective layer disposedbeneath the refractive layer, and wherein the refractive layer ischromatically dispersive.
 19. The method of claim 18, wherein thespatial light modulator comprises an array of cells having respectiveelectronically controllable refractive indexes, and whereinelectronically controlling a spatial light modulator during a firstperiod of time such that a refractive layer of the spatial lightmodulator has a refractive index that varies spatially across thespatial light modulator according to a controllable gradient compriseselectronically controlling the refractive indexes of the cells such thatrefractive indexes of the cells vary in a direction corresponding to thefirst specified direction and at a spatial rate of change correspondingto the first specified magnitude.
 20. The method of claim 13, furthercomprising: controlling, using an actuated stage, the location of theparticular region of the target relative to the optical system.